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FOREWORD 


This Handbook consists of three volumes of which this is Volume II. The 
organization of the Handbook segregates three major aspects of internal auditing. 
Volume I addresses the establishment and development of an effective internal 
audit function. Volume II provides guidance on implementing internal audit 
concepts and practices generic to any audit. Volume III is reserved for Internal 
Audit Guides providing guidance on the audit of specific activities. The first two 
volumes have been published separately. Volume III will be published in the form of 


individual Guides, as they become available. 


Volume I provides guidance on the development, approval and promulgation of an 
internal audit policy, on the objectives, responsibilities and composition of an 
internal audit committee, on identifying the audit universe and developing the 
long-term plan and annual schedule of internal audits, and on the management of 
an internal audit function. Volume II builds on the base provided by Volume I and 
provides guidance on planning and conducting each individual audit assignment 


including various concepts and approaches to be considered and utilized. 


The two-part presentation of Volume II is designed to segregate the discussion of 
the internal audit process (Part 1) and discussion of general internal audit concepts 
and approaches (Part 2), the latter of which is largely independent of the audit 


process. 


Part I consists of individual chapters covering each of the key elements of the 
Internal Audit Assignment Process, namely Assignment Planning, Review, Evaluation, 
Verification and Reporting. It includes appendices covering the special topics of 

risk analysis in assignment planning, an illustration of an indexing/cross-referencing 
system for working papers and one on controls testing verification programs for 


essential controls. 
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Part 2, Internal Audit Concepts and Approaches, covers a broad spectrum of generic 
internal audit concepts, methods and techniques. It includes Internal Audit 
Approaches: Objectives and Lines of Inquiry; Control: Concepts and Applications 
for Internal Auditors; Management Control: Concepts and Practices; Analysis 
Concepts and Practices for Internal Auditing; Audit Evidence; Auditor Judgment; 
Auditor-Auditee Relations; Communication Concepts for Internal Auditors; and 


Auditor Reliance. 


The list of subjects discussed in this volume does not attempt to cover all relevant 
internal audit concepts, methods and techniques, nor does it give all topics equal 
depth of coverage. This is by design. It is not our intention to duplicate existing 
audit literature where it is considered sufficient, but rather to supplement what 


exists in readily available form. 


In particular, statistical methods for auditors is well covered in text books and, 
therefore, is simply referred to in bibliographic terms in Chapter 4, Analysis 


Concepts and Practices for Internal Auditing. 


On the other hand, the topic of Computer Aided Audit Techniques (CAAT) is 
introduced in Chapter 6, Auditor Judgment, but not covered in great detail simply 


because the topic is too volatile at this point to capture in definitive terms. 


It is expected that, as internal audit concepts, methods and techniques evolve, 


some chapters will be added, (e.g. one on CAAT) and others will be revised. 


It may have occurred to some readers that there is a glaring oversight in this volume 
in that there is no chapter on EDP auditing. Again this is by design. The contention 
embodied in this volume is that all concepts, methods and techniques covered here 
are totally generic and, therefore, as applicable to EDP auditing as to any other 
"technical" area, (e.g. personnel auditing, auditing of records management, or 
auditing of line organizations, such as ocean sciences, environmental protection, 
health protection, etc.). Therefore, EDP auditing, like audits of other specialized 


areas, should be covered in Guides which will form part of Volume III. 
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PART ONE: THE INTERNAL AUDIT ASSIGNMENT PROCESS 
INTRODUCTION 
Purpose 


The purpose of Part | of the Internal Audit Handbook is to provide a framework for 
developing and conducting a typical audit assignment. It will assist the internal 
auditor in recognizing key elements of the process, their purpose and what techniques 


may be employed to successfully execute them. 


A pictorial overview of the relationship between the internal audit process and 
related issues is provided in Figure 1. Part | also attempts to demonstrate the 
nature of the process and the linkages between its principal elements. The concepts 
underlying the process and appropriate associated techniques are covered in Part 2 
of Volume II of the Handbook. 


The audit assignment process described here emphasizes planning and execution of 
review, evaluation and verification activities. These activities are aimed at 
determining if controls created by departmental or agency managers ensure the 
efficiency, economy and effectiveness of their processes and results. This is 
accomplished, in essence, by developing a predetermined control model, or modifying 
an existing one, in consultation with the auditee, and then assessing the auditee's 


actual control framework against the agreed-upon predetermined control model. 


Since the description of the audit assignment process presented here is necessarily 
general in nature, it cannot take into account all the specific or unique charac- 
teristics of different entities being audited. It does, however, provide a framework 
which should help to add a consistency and thoroughness of approach to the audit 


assignment process. 


The application of sound judgment and professionalism will continue to remain 


important ingredients for quality results. 
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THE INTERNAL AUDIT PROCESS 


(Relationship with Internal Audit Philosophy, 


Standards, Concepts, Methods and Techniques) 


Internal Audit Internal Audit Process 


— Philosophy — Assignment Planning 


— Principles — Review 

— Policy — Evaluation 
— Standards — Verification 
— Concepts — Reporting 


Internal Audit 
Effectiveness 


Figure | 


Internal Audit 


— Methods 


— Procedures 


— Techniques 
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The Internal Audit Assignment Process 


The internal audit assignment process involves a combination of sequential and 
iterative (i.e. looping back to previous steps) activities based on the following 
fundamental elements: planning the audit assignment, planning audit steps or 
programs (related groups of steps), gathering relevant data, deciding when data 
qualifies as audit evidence, analysis of evidence, verification of evidence, deciding 


on findings and conclusions and developing recommendations. 


For purposes of discussion, these fundamental elements are grouped into the 
following audit assignment phases: assignment planning, review, evaluation, 


verification and reporting. 


The audit assignment process necessitates on-going, continuous collaboration and 
communication between the auditor and the auditee if it is to be successful. 
Internal Audit is an advisory (staff) function. The internal auditor has no power to 
direct auditees, therefore, cultivation of good rapport is essential to an effective 
audit process. (Refer to Volume II, Part 2, Chapter 7, "Auditor - Auditee Relations" 


for an elaboration.) 


Assuming regular dialogue between auditor and auditee occurs, there should be no 


surprises either during the process or at the assignment's completion. 


The audit assignment process also requires reassessment at each stage in that each 
step should build upon or modify previous efforts to ensure that the audit process 
focuses on areas of greatest concern and does not pursue lines of inquiry that are 


not likely to be cost-effective. 


What Part 1 attempts to do is to guide the auditor through this process by focusing 
on the principal elements and their relationships. It highlights the key tasks to be 
undertaken, indicates where further planning or reassessment may be required and 


links the iterative and sequential steps into an integrated process. 
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It should be noted that while Part | walks through each of the five principal audit 
phases in the process, it is not so rigid that it requires each and every step to be 
done for every potential finding. For example, not all evidence requires verifi- 
cation, not all potential findings are reported, etc. Also, as already indicated, not 
all phases follow each other ina strictly sequential manner. For example, while an 
evaluation phase exists, it does not mean that evaluation occurs only at a specific 
point in time in the process. Evaluation is a continuous appraisal process. It 
overlaps all the other phases of the audit. The dynamics of the audit assignment 


process are illustrated in Figure 2. 


Figure 3 outlines the audit process as described in Part | and can be used as a 


reference to ensure that each of the critical steps of an assignment is addressed. 
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DYNAMICS OF THE AUDIT ASSIGNMENT PROCESS 


ASSIGNMENT PLANNING 


REPORTING 


Figure 2 
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CHAPTER ONE 


ASSIGNMENT PLANNING PHASE 


INTRODUCTION 


It is the responsibility of the director of internal audit to direct the preparation of 
the long-term and annual plans for the internal audit unit. The plans will include a 
broad definition of purpose, scope and objectives for each assignment, together 

with a forecast of the time and resources required to complete the assignment and 
the expected timing of the planned audits. The audit managers or team leaders in 
charge of audits or specific audits respectively are then responsible for translating 
the broad purpose, scope and objectives statements into action plans tailored to 


each audit assignment. 


Assignment planning is a progressive process which takes place throughout the 
entire audit. It must therefore be recognized that although the major assignment 
planning effort occurs at the beginning of the process, it is expected that the plan 
will be "fine tuned" during subsequent phases of the assignment. The extent of fine 
tuning will, of course, diminish as the auditors learn more about the nature and 


state of the auditee environment. 


We will now proceed to describe that phase of the audit process which is called 
assignment planning. It is a derivative of long-term and annual planning (discussed 
at length in Volume I, Chapter 3) and a prerequisite to the conduct of the audit of a 
specific audit assignment. It starts with familiarization with the audit entity and 


culminates in a detailed audit planning memorandum. 
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SECTION ONE: PRINCIPAL COMPONENTS 
The principal components of the assignment planning phase are: 
e development and definition of purpose, scope and objectives (I.1)!; 


® preliminary identification of significant audit areas and development of 


audit approach (i.e. Lines of Inquiry) (1.2); 
® communication of audit purpose, scope, objectives and approach (I.3); 


8 identification of staff and other resources required, and assignment of 


responsibilities (1.4); 


a preparation of time budget and work schedule (i.e. a project plan) (1.5); 
® planning of audit working papers (1.6); 
@ preparation and approval of an assignment planning memorandum (I.7); 
and 
* communication with auditee prior to commencing the audit fieldwork 
(1.8). 
l The numbers in parentheses refer to the components of the audit process as 


shown in Figure 3. 
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Development and Definition of Audit Purpose, Scope and Objectives 


Since the annual audit plan outlines only the broad purpose, scope and objectives of 
each audit assignment, the team leader must establish, early in the planning phase, 

a more specific definition of the purpose and scope of the assignment and more 
precise audit objectives. The usual principal purpose of any audit is to determine 

the extent to which essential control objectives are met by the audited organization. 
To establish a meaningful specific audit purpose, scope and objectives, the team 
leader requires the following: 


® information on/or a decision on the type of audit planned; 

r a good knowledge of the audit entity's administrative infrastructure; 

e a good knowledge of the delivery systems of audit entity's operations or 
activities; 

a good knowledge of the audit entity's long-range and current plans; 

® a predetermined control model for the audit entity; 

© knowledge of the audit entity's current risk exposures; 

® knowledge of other planned audit and quasi-audit activities in the same 


area and timeframe; and 


& sources of acceptable audit criteria to be applied. 


It may be seen from the foregoing that the team leader, in formulating the purpose, 
scope and objectives, must develop a good working knowledge and understanding of 
the audit entity and then, using that knowledge, develop the specific requirements 


of the particular audit being considered. 
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In developing an understanding of the audit entity it is not likely that the team 
leader can gain an intimate understanding of all its activities at this early stage of 
the assignment (this is particularly true if it is the first audit of that unit). Rather, 
an overall appreciation of the nature, characteristics and boundaries of the audit 
entity is all that is expected. To obtain such an appreciation, the team leader will 


perform, as a minimum, the following tasks: 
e identify and examine the entity's mandate and related authorities; 


8 identify and examine significant legal, financial and regulatory constraints 
including, as sources, such documents as the Main Estimates, legislation, 


regulations and central agency policies; 
® examine corporate objectives, plans, policies and directives; 


% develop an understanding of the corporate structure and activities by 
examining corporate organization charts, position descriptions and 


relevant in-house correspondence and documentation; 


8 identify associated priorities and problems through reference to avail- 
able accountability models and systems descriptions, previous internal 
audit files and reports, central agency evaluation reports and internal 


program review and evaluation reports; and 


e review significant financial and other operating data including long- 
term and annual budgets and related variance reports, management 
reports, person-year allocations, performance measurement reports and 


transaction volumes, where available. 


The general purpose of the audit is derived from standard No. if which reads: 
"Departments shall have an independent internal audit function that carries out a 
systematic review and appraisal of all departmental operations for purposes of 


advising management as to the efficiency, economy and effectiveness of internal 


2 Standards for Internal Audit in the Government of Canada, 1982. 
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management policies, practices and controls." The wording is adjusted, however, 
to suit the audit unit. On the other hand, the detailed purpose and scope of the 


audit assignment is largely determined by the type of audit undertaken. 


As indicated in the foregoing, it is important to identify clearly the type of audit 
since this will significantly affect the definition of audit purpose, scope, objectives 


and lines of inquiry.? The principal types of audit are: 


Responsibility Centre Audit 


(a) Purpose: A responsibility centre (RC) audit is undertaken for the purpose of 
providing managers with advice on all the important operations and activities under- 
taken by that RC (whether program or administrative), the management of that 

RC, and the impact of functional direction (program and administrative) on the 
activities of that RC. 


(b) Scope: The scope of an RC audit is determined solely by the organizational 
boundaries of the RC itself. 


Organizational Audit 


(a) Purpose: This is an audit which provides management with advice on an 


organizational unit of one or more RCs or parts of RCs. 


(b) Scope: The scope is determined by the combined boundaries of the RCs, or 


parts of the RCs, included in the audit unit. 
Function Audit 


(a) Purpose: This is an audit which provides advice to management on an 
organizational unit which provides functional direction to other organizational 
units and includes both program and support (e.g. finance, administration, personnel, 


official languages) functional activities. This functional direction will usually be 


3 The objectives and lines of inquiry, for each general type of audit outlined 
here, will be discussed in more detail in Volume II, Part 2, Chapter 1, "Internal 
Audit Approaches: Objectives and Lines of Inquiry". 
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provided through the issuance of formal policies and directives. Typically, an audit 
of the effectiveness of adherence to such directives will form one component of a 


function audit. 
(b) Scope: Functional audits are usually performed on two levels: 


- audit of the organizational unit which is the source of functional direction; 


and 


- audit of adherence to policy and directives in all or a sample of the 
organizational units which are subjected to functional direction, and the effect 


of the policy and directives on those units. 


Program Audits 


A program audit is a responsibility centre audit of the RC, or RCs, charged with 
the overall responsibility for the program or activity (e.g. a branch) and, where 
applicable, all or a sample of the program-related operations of the responsibility 


centres (e.g. regions, field offices) involved in the delivery of the program. 


Systems Audits 


Although the term "systems audit", if one were to take the most generic definition 
of the word "system", would include all types of audit unit described above, the 
term as used here is meant to include only systems in the narrower sense as used by 


EDP or systems and procedures staff. 


(a) Purpose: A systems audit is an audit which provides management with advice 


on a system used by any functional or line unit, or any combination of the two. 


(b) Scope: This type of audit is similar in nature to the functional audit in that it 


is carried out by performing: 


- a review of the organizational unit which is responsible for the overall 
design, development and maintenance of the system (the administering 


unit, typically the EDP group); 


2 
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- a review of the organizational units which host (own) and use the system 
(i.e. have input and output interface with the system); and 


- a review of a sample of the users of the system. 


Pre-implementation Audit 


(a) Purpose: This is an audit of major systems under development, including 
legislation, policies and procedures, information systems, program delivery systems, 


contracts, etc., prior to their implementation. 


(b) Scope: This type of audit may consist of one or both of the following: a 
relatively limited-scoped audit, principally, with the auditability and adequacy of 
controls built into the systems to be implemented; and an audit that may include in 


its scope an assessment of the systems development process itself. 


Special Audit 


(a) Purpose: The special audit is usually performed at the request of management, 
normally to deal with unforeseen situations, policy developments or other senior 


management concerns, including suspected fraud or defalcation. 


(b) Scope: The special audit is narrow in scope and will usually be executed in a 


comparatively short time frame. 


A review of related previous, concurrent or planned audits (internal, external, other), 
program evaluation and other similar control-oriented assignments provides a basis 
for determining the extent of any likely overlap with the current internal audit 
assignment. Previous, concurrent or planned audit activities may significantly 

alter the scope of the present assignment or the required depth of coverage. For 
example, the audit of a particular function may be split into several components 


such as: 


& a review of process controls within the function; 
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® a review of management controls within the relevant responsibility 
centre; 
2 a review to ensure compliance of other responsibility centres with 


functional direction provided by a functional responsibility centre in 


order to assess its effect on the organization. 


In these situations, separate audit teams may be assigned to examine each component. 
Clearly there will be a necessity for careful planning to consider the extent to 
which resources may be utilized to achieve objectives common to more than one 


component. 


For most types of audit (excluding special audits) the audit objective is to provide 
management with assurance that information it is receiving as to the state or 
performance of the audit entity under review is according to management's 


expectations. This requirement is reflected in the predetermined control model. 


This assurance is provided in terms of two broad and inclusive categories of perfor- 


mance objectives: 


(1) that predetermined results are being achieved (economy, efficiency and 


effectiveness); and 


(2) that prescribed infrastructure (e.g. delivery systems/administrative systems/ 


organization structures) is performing as intended. 


The need for the first objective is fundamental; the second is subsidiary to the 
first, in that not only do you want to know that objectives are being met now, but 


that they will continue to be met. 


In addition to performance objectives, the predetermined control model will reflect 


required physical controls, to the degree that they are specified by higher authority. 
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Finally, to enable the auditor to judge whether or not objectives are being met, the 
model provides criteria. Since the level at which criteria are defined must correspond 
to the level of their corresponding objectives, little detail can be provided at this 
time aside from asserting that they exist explicitly to the degree that required 
infrastructure and results are pre-defined. Therefore, it is not until we contend 

with a specific audit entity, with specific, prescribed infrastructure and pre-defined 
results, that detailed criteria can be defined. The foregoing presupposes that the 
audit assignment planner is starting out with a predetermined control model already 


available. Where it is not, it will have to be developed.’ 


Performance criteria for control evaluation are required at two levels: (1) for 
compliance review, we need criteria for recognizing that a required control exists 
(including all its required elements); and (2) for substantive testing, we need criteria 


for determining if the control is effective.” 
Sources of performance criteria include: 


2 audit guides developed under the auspices of the Interdepartmental 


Advisory Committee on Internal Audit; 


@ criteria developed for other similar audits; 
Y central agency policy groups or their output; 
# comprehensive auditing criteria for departments and central agencies 


developed by the Auditor General of Canada; 


a The ideal situation is where a predetermined control model has been developed, 
jointly with management, in a time frame which is totally independent and 
preceding the audit assignment time frame. If a predetermined control model 
is not already available, the second preferred process is one where the model 
is developed during the review process, concurrently with review and 
documentation of existing systems, etc., but by an independent team which 
includes management representation. It is recognized, however, that for 
pragmatic reasons the preferred approach will not always be possible to 
implement in actual practice. See Volume II, Part 2, Chapter 2, "Control: 
Concepts and Applications for Internal Auditors" for a further discussion of 
the predetermined control model. 


> See Volume II, Part 2, Chapter 2. 
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® pronouncements of professional organizations; 
3 literature on the subject under examination written by qualified 
professionals; 
a input from specialists active in the area; and 
co management's own performance criteria. 


The updating (development, if not already available) of acceptable objectives or 
criteria is an integral part of the development and maintenance of a predetermined 
control model. This activity is performed mainly in the review phase; the results 
will form the basis for audit activities to follow. It is, therefore, essential that 
sources of criteria be identified as early as possible in the audit assignment and 
that agreement be reached with auditee management on the acceptability of the 
identified sources. This will assist the team leader in planning for the review phase 
and it should reduce the potential for disagreement with the auditee about audit 


findings. 


The starting point for an assignment is normally the broad general statement of 
purpose, scope and objectives contained in the annual plan. In the steps outlined in 
Part 1, the team leader will develop and modify those general statements to suit 
the particular circumstances of the entity under examination. Therefore, it is 
essential for the team leader, once the specific audit purpose, scope and objectives 


have been developed, to reconcile them to the requirements of the annual plan. 


In many instances this will involve simply a direct comparison to ensure that they 
reflect the intentions in the overall plan. However, in a number of cases the team 
leader may discover in the course of assignment planning that the terms of ref- 
erence originally envisaged are either inadequate or inappropriate to the circum- 
stances of the audit entity. The team leader may conclude, for example, that the 
scope requires significant expansion, amendment or indeed that the assignment is 
unjustified in its present form. Consideration of these matters should be documented 
by the team leader together with decisions taken and the assumptions underlying 


such decisions. 
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In the worst case this may result in the audit manager or director having to adjust 


the annual, or even long-term, audit plan. 


Preliminary Identification of Significant Audit Areas and Development of Audit 


Approach 


To identify significant areas for examination the team leader must become familiar 


with all major operations of the entity and with their respective risk potential. 


This process will normally incorporate some or all of the following: 


describing in outline form the program delivery system along with its 


expected results; 


identifying the controls in place to measure and report on the 


performance of the program delivery system; 


identifying the controls in place to measure and report on the economy, 


efficiency and effectiveness of the program or activity; 


describing in outline form the administrative support systems along 


with their expected outputs; 


identifying the controls in place to measure and report on the 


performance of the administrative support systems; and 


identifying the controls in place to measure and report on the economy, 


efficiency and effectiveness of administrative support. 


These include: 


identifying the nature and extent of EDP systems; 


identifying mechanisms for the protection of assets; 
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6 producing overviews of the key management functions and their inter- 


relationship; and 


2 outlining the management of financial, human, physical and information 


resources, 


The inforrnation gained will provide broad insight into the nature and state of the 
audit entity. The specific activities to be given the most attention in the audit 


must then be identified. These will be determined in terms of the following factors: 


e materiality 

e management concerns 

% inherent risk 

8 control risk (relative risk) 

F) risk potential (risk of loss or opportunity cost potential!) 

@ weaknesses identified by previous audits or program reviews 
7 cost-benefit implications. 


The Concept of Risk - Its Dimensions 


To make decisions on which auditee areas should be given the most attention it will 
be necessary for the auditor to assess the risk of not giving it attention; i.e. assess 
the area's risk potential. In managers! terms - What are the repercussions of being 


out of control in a specific area or activity for which the auditee is responsible? 


The foregoing "factors" are all components of the risk assessment process. In 
what follows, the relative contribution of those factors, to the assessment of risk 
potential, will be discussed and an illustration will be provided of the use of risk 


potential in deciding on how to allocate available resources to auditee areas of 
responsibility. 


J 


Internal Audit Handbook 
Volume II, Part 1 
Chapter 1, Section | -19- 


In Volume II, Part 2, Chapter 5, "Audit Evidence", the concept of materiality is 
discussed initially in terms of its external audit origins; however, this discussion is 
expanded upon through a quote from Anderson® as follows: "In internal auditing ... 
an item would be considered material if an error in it (or its complete omission) 
would cause prudent, intelligent information users to change decisions that they 
might otherwise make on the basis of information provided by and about the auditee." 
Since internal auditors are mirrors to management of management's control systems, 
which in turn are important inputs to management's decision-making process, the 
above definition of materiality is equally applicable to managers and auditors. As 
used here, the term "materiality" reflects both aspects of risk; that is, size of 


potential risk and its probability of occurrence. 


Inherent risk refers to the natural error rate of the process under audit. No process 
is error free and reducing the natural error rate may be prohibitively expensive. 
This fact must be taken into account in any evaluation of risk potential. Assuming 
that the inherent risk is tolerable, risk potential then has to do with an incremental 
error (i.e. the risk of tolerating errors which exceed the inherent process error rate 


by a material amount) not with the absolute error. 


Control risk is the risk of not detecting material deviations of the actual error rate 
from the acceptable error rate. Control risk is a measure of confidence that the 
manager, or any observer, has in the control system rather than a measure of the 


absolute risk of the system under control. 


Finally, and most importantly, we will discuss the concept of risk potential as it 
applies to audit assignment planning. The concept of materiality in terms of 
potential error in an assertion is helpful in discriminating between those areas most 
likely to have material variances and those that are not; however, for those cases 
where resources are insufficient to cover all areas where material error is likely to 
occur, additional criteria will have to be developed, by the audit team leader, for 


resource allocation decisions. 


6 R.J. Anderson, The External Audit. 
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The preceding scarce-resource-allocation decision can be stated in the 

following form: How much are we willing to pay for the additional information 
that we are likely to obtain from an internal audit of the area in question (in terms 
of the key decision(s) to be made in that area)? If the projected cost of information 
we expect to gain is less than the potential improvement in the subject decision- 
making process, then the audit resources should be allocated accordingly, in 


proportion to the improved results expected. This is an opportunity cost decision. 


The criterion most appropriate to this particular opportunity cost decision is risk 
potential, in either potential real loss or potential opportunity loss terms. The 
methodology for assessing risk potential, in these terms, is developed in Volume I, 
Chapter 3 and Appendix C of the Internal Audit Handbook. An illustration of its 
use is provided in Appendix A to this Part. 


From the foregoing, and a review of Volume es it will be evident that the key factor 
in resource allocation is that of "risk potential". All of the other factors are utilized 
(directly or indirectly) in the determination of risk potential through its two major 


components, i.e. "materiality" or "probability of occurrence"). 


Having identified potential risk areas for examination, the team leader must develop 
an overall strategy which will ensure that all the most significant audit areas are 
covered in the most effective and efficient manner. The audit strategy should be 


such that it encompasses: 


8 adequate coverage of all major audit areas; 
@ an approach consistent with the scope and objectives of the audit; and 
© appropriate extent and depth of testing. 


Z Internal Audit Handbook, Volume I, Chapter 3 and Appendix C. 
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The critical factors in developing the overall audit approach will be the scope of 
the audit and the audit objectives established. At this stage of the audit it is not 
expected that detailed procedural guides will be prepared but rather that the general 


thrust of the audit approach will be determined and broad lines of inquiry established. 


The depth of the audit and the extent of testing will be determined largely by the 
audit objectives proposed, needs of the potential users, results of the review phase 
and availability of time and human resources. At this stage the team leader should 
be in a position to outline the depth of coverage and estimate the proposed extent 
of testing in general terms. Further development of these factors will take place 
during the review and evaluation phases of the assignment and will crystallize in 


the verification audit plan. 
Communication of Audit Purpose, Scope, Objectives and Approach 


Once the team leader has defined the purpose and scope parameters, established 
specific audit objectives and developed a general audit approach, it is usually 
desirable to discuss these with the audit manager and auditee. Normally, there will 
be a preliminary discussion between the team leader and the audit manager followed 


by a review with the auditee and a final approval by the audit manager. 


Discussion of the general goals and thrusts of the audit will likely be incorporated 
in the agenda for a preliminary meeting with the auditee. Normally, the team 

leader will outline to auditee management the audit purpose, scope and objectives 
developed, the predetermined control model (if already developed), the sources of 


criteria to be used and the general audit strategy proposed. 


In addition to keeping the auditee fully informed, this discussion will also seek to 
elicit any concerns or disagreements the auditee may have with the proposed frame- 
work. Where these are significant, it may be necessary to re-evaluate or modify 
the audit approach to incorporate management's suggestions and concerns. Only 
after the preliminary meeting and consideration of auditee input will the team 
leader be in a position to finalize the scope and objectives of the assignment for 


review with the audit manager. 
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Review procedures may vary from one audit group to another, but generally will 
take the form of a memorandum from the team leader to the audit manager with a 


copy to the director, outlining: 


e the proposed purpose, scope, audit objectives and related source of 
criteria; 

) the proposed lines of inquiry; 

® the degree of consistency with long-term and annual plans; and 

& the assumptions and proposals for amendment where it is judged that 


the original terms of reference are inadequate. 


At this stage, the audit manager will usually discuss the proposed audit approach 

with the team leader to ascertain that the audit coverage is appropriate and adequate, 
that appropriate auditee input has been incorporated into the plan and that the 

audit approach is likely to be both feasible and cost justifiable. If the audit manager 
is satisfied on these points, approval of the preliminary plan may be expected and 

the team leader can then proceed with the identification and allocation of resources 


and with scheduling. 
Identification of Staff and other Resources Required and Assignment of Responsibilities 


In most instances, the annual plan will contain some indication of the staff resources 
to be assigned to each audit assignment as well as the dates within which these 
resources will be made available to the team leader. These are constraints which 
the team leader must take into account when determining the staff resources 
required for the successful completion of the audit and the deployment of those 
resources to ensure they are utilized to best advantage. This process will typically 


fall into three stages: 


® identification of quantity and quality of staff resources required, 


including the identification of necessary specialist skills; 
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e matching the staff resources required to the resources available; and 
e assignment of responsibilities within the audit team. 


The team leader must determine the skills required and the appropriate experience 
levels necessary. The techniques for arriving at this determination will depend on 


the auditor and the circumstances, but the following general approach may be useful: 


® develop a matrix indicating audit areas to be covered against resource 


skills necessary to carry out the audit satisfactorily; and 


w assess the expected time commitments for each resource identified on 


the matrix. 
This exercise will provide an indication of: 
* the quantity and quality of resources required; 


& whether there is a requirement for specialist skills and, if so, in which 


area of the audit they will be needed; 


® the time requirements for each resource and for the audit as a whole; 
and 
@ the extent to which the resource requirements match the resources 


assigned to the audit in the annual plan. 
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A sample resource requirement matrix is outlined in Table 1. 


Table | 


Sample Resource Requirement Matrix 


Skills Required 


Audit Team Auditor Auditor EDP Other Time 
Issues Leader II I Specialist Specialist Required 
iP 20 30 30 80 
Zz. 40 60 100 
as 40 30 70 
4, 10 60 70 
ie 30 70 100 
6. 25 25 50 100 

Time . 
Required 165 115 120 70 50 520 


In developing the total resource requirements for the assignment the team leader 
will need to consider other resource requirements to cover such items as travel 


time and costs, computer time and other related assignment expenses. 


It should be emphasized that development of resource requirements is an inclusive 
process incorporating all resources necessary to complete the assignment (human, 


physical and financial). 


From the matrix or other techniques employed to determine appropriate staff 
resources the team leader should be able to identify any requirement for specialist 
skills. Having defined the requirement, the team leader must then consider the 
availability of such skills. Where the appropriate specialists are available within 
the audit branch, the team leader will request their deployment to the audit team 


for the period required if this has not been provided for in the annual plan. 


Where the required specialist skills are unavailable within the audit group, the 


team leader must consider alternatives: 
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® obtaining the skills from elsewhere in the host organization (i.e. 


temporary assignment or secondment); 


@ obtaining specialists externally (i.e. by contracting); 
® stretching existing skills; or 
© altering the scope of the assignment. 


These alternatives represent major project decisions and should be referred to the 
audit manager or director for consideration. The responsibility of the team leader 
is to review the requirement and develop proposals best suited to the achievement 


of the audit objectives, or revise the objectives. 


All of the alternatives require additional planning. In particular, the use of 
specialists complicates the normal audit process. A complication which affects the 
first three alternatives is that of appropriate division of labour. Then there are 


complications which are unique to the alternative chosen. 


The division of labour issue is one of integrating audit and non-audit expertise so 
that optimum input from each may be brought to bear on the result. There are two 


key ways in which experts may be employed to advantage. 


If the expert is a subject matter or function expert (e.g. an expert recruited from a 
specific program such as Environmental Protection or from a specific function such 
as EDP or Personnel), that expertise may be used to advantage in: the creation or 
updating of the predetermined control model and in its validation; the development 
of audit verification programs for substantive testing (determining the objectives, 


key variables and parameters); and the interpretation of the results. 


Technique-oriented experts (e.g. statisticians, econometricians, operations research 
specialists) could be used directly in the development stages and indirectly (for 
advice on the use of specialized audit methodology and techniques) during the rest 


of the audit assignment process. 
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Complications which are unique to the alternative selected include the following: 


(1) In-house specialists would have to be assigned in such a way that their special 
skills are used to advantage and the effect of their lack of audit skills is 
minimized. Basically this means that they are used primarily in the methodology 
and techniques-bound phases of the audit process, particularly in the "review" 


process. 


If they are expected to become a regular part of the audit cadre in due course, 
then in addition to the productive part of their assignment there should be 
training or apprentice activities built into the schedule, where the specialists 
would be assigned to an auditor as an assistant for development purposes and, 


possibly, scheduled for formal training courses as well. 


(2) Where the specialist is seconded from a program or functional group else- 
where in the host organization, then the assignment would have to be arranged 
so that conflict of interest is avoided and the future of the secondee is not 
adversely affected by the assignment. There are two ways in which these 


possible effects may be avoided: 


(i) avoid using secondees on audits of their respective home 


organizations, and 


(ii) use them only in the review phase, as opposed to the evaluation 


and reporting phases, of the audit process. 


(3) In the case of resources on contract, there is the problem of whether the 
specialist is to work under supervision of the in-house team leader or 
autonomously. If the specialist works under supervision of the team leader, 
then the process can be relatively informal and treated similarly to the case 
of the audit group, in-house specialist. Where the specialist works 
autonomously, under supervision of the parent accounting or consulting firm, 
the terms of reference of the contracts will have to be much more precisely 


defined. 
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When resource requirements and availability have been established the team leader 
must ensure that audit responsibilities are assigned clearly and that resources are 
used to maximum advantage. 


Preparation of Time Budget and Work Schedule 


The team leader must develop an overall workplan during the assignment planning 


phase which will: 


e provide a framework for work scheduling and staff management; 

% reflect the audit approach and the emphasis therein; 

® be sufficiently flexible to allow for the most efficient use of resources; 
® provide a mechanism to measure the progress of the audit through mile- 


stone dates; and 


% provide information for future audit planning. 


To prepare a workplan which will satisfy these requirements and provide the team 
leader with a useful audit management tool, a plan must be developed which is 

more than a simple time budget and a list of audit milestones. It is essential that 
the human and time resources available be integrated with the timing and scheduling 
constraints imposed by the nature of the audit and the requirements of the auditee. 
It may be necessary not only to allocate time to specific tasks and staff, but also 

to establish when that time should be spent on a calendar basis, so as to satisfy 

both scheduling (auditor and auditee requirements dealt with) and milestone 


requirements. To achieve this, the team leader should: 


® define clearly the total time required for completion of the assignment; 


e establish accurately the availability of required resources; 


8 ascertain carefully the key audit milestones; and 
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® prepare an audit time and scheduling workplan, possibly in matrix form, 


setting out the audit areas to be covered against the available resources. 


The team leader will thus determine the total estimated time required for the 
completion of each audit area and allocate that time among the staff assigned 
for audit work in each area. The resource identification matrix suggested earlier 


in the planning phase will usually serve as a basis for this plan. 
Once the time budget has been developed, the team leader will need to refer to the 
key milestones and scheduling constraints to determine which areas, if any, require 


modification. 


From this exercise the team leader will derive an audit workplan which will indicate 


clearly: 
© total time required; 
® time allocated to each audit component; 
@ time allocated to each staff member; and 
9 dates between which the audit work will be performed. 


Normally, this workplan will be incorporated in the assignment planning memorandum 


and will be in a form similar to that outlined in Figure he 


8 See Volume I, Appendix A, for a departmental example of an audit assignment 
status report. 
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WORKPLAN 


A. Human Resources 


aa Te 
Peale reese 


Legend: 
Proj. = Projected 
Act. = Actual 


Related audit area ex yn 3 


B. Other Resources 


Resource 


1. Travel costs 


2. Computer time 


3. Other 


Figure 4 
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Planning of Audit Working Papers 


The team leader should carefully plan and prepare the working paper file for the 
fieldwork. In this regard, the table of contents is of particular importance (see 
Appendix B at the end of Part | for an example of a Working Paper Index). It should 
make provision for certain standard matters to be covered as the assignment 
progresses and establish the control reference for the working papers. Also, it will 
provide a check to ensure that all areas of the audit are covered and that all 


necessary documentation is on hand when the audit team goes to the field. 


If the working papers are planned carefully in accordance with the objectives of 

the audit and the issues perceived to be important, they should correspond closely 
with the broad framework of the audit report, thus facilitating report drafting and 
referencing. (See Policy Interpretation Notice 1983-02 for a more detailed treat- 


ment of the subject of Working Papers.) 
Preparation and Approval of an Assignment Planning Memorandum 


The assignment planning memorandum is the principal product of the planning phase 
of the audit. It should provide a documented summary of planning information, 
audit considerations, assumptions and decisions taken. Also, it will constitute the 
overall audit plan for the assignment, serving as the basis to plan and conduct sub- 
sequent phases. The planning memorandum will form part of the working papers 
for the audit and it is essential that it be approved at the appropriate level before 


further work on the assignment begins. 
An assignment planning memorandum should include as a minimum: 


® a summary of the purpose, scope and objectives of the assignment and 


any limitations placed thereon; 
© an overview description of the audit entity encompassing: 


- legislative authorities and mandate 


») 
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- key objectives and goals 


- resources employed (human, financial and physical) 


= key organizational and operational issues and constraints 


- principal information and operational control systems 


- principal management control mechanisms 


- significant changes in operations, system and influences over the past 


two years or since the last audit 


- known or expected future influences on the audit entity, and 


- key assumptions made; 


8 principal issues and significant audit areas; 


® audit strategy decisions including the audit approach and general notes 


on audit technique decisions; 


® basic objectives and associated sources of criteria; 

@ outline of audit report format(s) taking into account levels of reporting 
required; 

@ resource requirements including identification of specialist skills; 

) time budget and critical milestone dates; 

@ assignment of staff responsibilities; 

® any other significant considerations or outstanding issues; and 
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* confirmation of approval to proceed. 


The assignment planning memorandum will be submitted to the audit manager in 
charge of the assignment for review. After discussion of details with the team 
leader, the audit manager will normally conduct a review with the internal audit 
director outlining the principal features of the assignment plan. Provided all require- 
ments of the manager and director are satisfied, approval and authority to proceed 


with the fieldwork will be provided. 


Communication with Auditee Management Prior to Commencing the Audit 
Fieldwork 


Preliminary contact with auditee management during the planning phase is a critical 
dimension of the audit assignment. Typically, the contact will take the following 


form during the assignment planning phase: 


Advance Letter to Auditee 


The team leader in charge of the assignment will draft a letter(s) to be signed by 
the director of internal audit, addressed to the manager(s) of the unit to be audited. 
Copies to the senior executive accountable for the unit may also be required. This 
letter will normally be dispatched well in advance of the scheduled commencement 


date and will outline: 


& the purpose of the audit; 

° the scope, in general terms; 

® the approximate time schedule and key milestone dates; 

e the files, documentation and schedules likely to be required; and 


e accommodation and facilities requirements, if necessary. 


6, 
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Preliminary Meeting with Auditee Management 


The preliminary meeting with auditee management should be an informal meeting 
between management of the unit to be audited and the team leader. It will normally 
be appropriate for relevant members of the audit staff to attend. Depending on 

the nature of the assignment, it may well be appropriate for more than one level of 
auditee management to be present at this meeting. It is also worth noting that 
more than one meeting may be necessary in certain circumstances. The agenda 


will typically include: 


e the broad purpose of the audit and, in general terms, the intended audit 
approach; 
% notification to auditee staff of the audit team's requirements to access 


files and related documentation; 


e discussion of timing requirements, milestone dates and any confirmation 


and physical inspection arrangements which may be necessary; 


® general discussion of the activities of each of the elements of the unit 


to be audited; 


& preliminary identification of areas perceived by the auditee as problems 


which may warrant a more concentrated audit effort; 


e discussion of material changes in systems, operations or personnel; and 


® a request for a typical list of documentation on the auditee area (a list 


of required documents should be provided to the auditee at the meeting). 


This preliminary meeting will be held prior to the final determination of 
activities to be audited and resources required and will form a critical input 


to that process. 
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Presentation of Audit Plan to Auditee 


Presentation of the audit plan to the auditee is normally arranged following 
completion and approval of the assignment planning memorandum containing the 
overall audit plan. This may take the form of an oral or written presentation or a 


combination of both. The presentation should cover the following matters: 


6 clear statement of audit scope; 

@ brief overview of the audit entity; 

® major objectives and related sources of criteria; 

e the general audit approach and indications of the techniques to be 
employed; 

« principal issues identified and significant audit areas; 

® critical timing requirements and milestone dates; 

8 the reporting procedure, including possibly an outline of the report and 


an assurance that the draft report will be discussed with the auditee 
management prior to its release to senior management and the audit 


committee; and 


% in cases where a predetermined control model is not already available, 
the auditee's support (including dedicated resources) may be obtained at 


this time for participation in its development. 
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SECTION TWO: PRINCIPAL PRODUCTS 


The principal products of the assignment planning phase are: 


® statement of audit scope and objectives; 

e overview of unit to be audited; 

* resource requirement workplan (time, dollars, people, facilities); 
e assignment planning memorandum; and 


e briefing for auditee management. 
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CHAPTER TWO 
REVIEW PHASE 
INTRODUCTION 


Once the planning of the assignment has been completed, the audit team normally 
undertakes a detailed review of the audit entity to expand its understanding of its 
principal features and problems. The review phase of the audit is devoted to the 
gathering and validation of more detailed information concerning the entity's 
programs, objectives and activities, to the development or updating of the 
predetermined control model and to the identification of existing controls and 


potential control weaknesses. 
SECTION ONE: PRINCIPAL COMPONENTS 
The principal components of this phase are: 
® expansion of the auditor's understanding of the entity (I1.1)75 
ae documentation of processes and systems (descriptive model)(II.2); 


e validation of data (II.3); 


% development or updating of the predetermined control mode! (II.4); 

@ comparison of descriptive model vs. predetermined control model (II.5); 
and, 

8 preparation of control registers (II.6). 


(Note: to the degree that this plan delineates the procedures to be followed 
by the auditor it, along with an expansion of steps II.1-II.5, comprises the 
front-end of what is often referred to as the "audit program". The remaining 
portion of the audit program would reflect procedures for verification and 
cause-effect analysis activities. (See Appendix C at the end of Part | for an 
example of an audit program.) 


9 The numbers in parentheses refer to the components of the audit process as 
shown in Figure 3. 
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Expansion of the Auditor's Understanding of the Entity 


Starting from the base developed in the planning phase, the auditor should review 


the information available carefully and comprehensively. This will enable the auditor 


to direct the knowledge gained towards the determination and investigation of 


significant issues and to determine the existence of required controls. 


Much of the information required for this review will be available in the planning 


files or in previous internal audit files. What is required during the review phase is 


an expansion of this base of information in order to gain a more comprehensive 


understanding of the audit entity. However, this expansion process must be cost- 


effective. 


The following may provide useful sources of reference for general information:!2 


information gathered and documented during the planning phase; 


internal management reports and other relevant published 


communications of the audit entity; 


selected interviews with auditee management to obtain perspectives 


and concerns regarding specific items; 


documented policies and procedures applicable to major programs, systems 


and controls; 


studies and reports of internal/external evaluation groups; and, 


personal observation and assessment of the audit environment. 


10 See Volume 11, Part 2, Chapter 5, "Audit Evidence" for more extensive 
treatment of the subject. 
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® Additionally, the review of available information should incorporate a further study 


of pertinent legislation and regulations, as required, to determine, clarify and 


confirm: 
e the purpose, scope and objectives of the audit entity; 
° the method of conduct, resourcing and control of activities; and, 
* the nature and extent of the entity's authority and responsibility. 


The organizational policies (departmental, branch, etc.) also should be reviewed 


carefully to ensure that: 


* departmental policies conform with applicable legislation and central 


agency, guidelines, policies and regulations; and, 


& policies are appropriate for efficient, effective and economical conduct 
» of authorized activities. 


Since much of this review will have been performed at a preliminary level during 
the assignment planning phase, a main objective of the review phase is to select, 
broaden or further develop the available sources of information to ensure a sound 


basis for the subsequent detailed analysis and evaluation. 


Internal Audit Handbook 
Volume II, Part | 
Chapter 2, Section | - 40- 


Development or Updating of the Predetermined Control Model 


A predetermined control model shows the significant points in a system's processes 
and the key results that should be subject to control; it also provides the objectives 
of the controls and the criteria used by the control in assessing performance. The 
need for a control in any particular location is determined by the degree to which 
management is willing to risk unwanted and undetected performance at the potential 
control point. (For a more detailed discussion on control concepts, refer to 


Volume II, Part 2, Chapter 2 of the Internal Audit Handbook.) Sources of criteria 


can be: 

® legislation governing the operations and the outputs of the audit unit; 

® central agency policies and directives which impact on the various 
administrative responsibilities of the auditee management (finance, 
EDP, personnel, official languages, etc.); 

® departmental policies and procedures (including performance indicators 
or standards) which affect the management and activities of the audit 

unit; 
@ departmental plans, and associated key results indicators; and 
® sound management principles and practices. 


Ideally, the predetermined control model would have been developed as an independent 
exercise (jointly with auditee and central agency managers/experts, as required) 


prior to the audit assignment. 


If a predetermined control model has already been developed as part of a central 
agency or departmental audit guide, it should be used as a starting point and 
modified, if need be, to suit the needs of the audit assignment. This will help to 


ensure the integrity and cost-effectiveness of the audit process. 
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In developing a new predetermined control mode! or modifying an existing one, the 
auditor will want to ensure that the model is appropriate to the characteristics of 
the audit entity. On the other hand, it is equally essential that there be no 
weakening of the auditor's objectivity (i.e. that the predetermined control model is 
not simply a reflection of the auditee's actual control framework). One way of 
minimizing bias is to segregate responsibility for the documentation of the system 
in existence and the development of a control model against which to evaluate that 
system. Where such segregation is planned, the team leader will often assume 
responsibility for the development of the predetermined control model and will 
assign responsibility for the existing system's documentation to the audit staff, 


usually under direction of a delegated team leader. 


Finally, it is important that the team leader seek and obtain the auditee's acceptance 
of the predetermined control model and its embedded criteria. This will prove to 

be very useful in gaining the auditee's acceptance of subsequent audit observations 
and conclusions. If the auditee management has accepted the sources of criteria 
identified in the planning phase, then it is quite likely that the predetermined 

control model will be equally acceptable. 


Documentation of Existing Organization Structures and Program/Support Delivery 


Systems 


Modelling techniques! | are usually appropriate for the recording of process or 
system structures. There is a wide range of modelling techniques available to the 


auditor including: 
® narrative systems descriptions; 


e organization charts and responsibility diagrams or other pictorial 


methods of modelling organizations; and, 


11 See Volume II, Part 2, Chapter 4, "Analysis Concepts and Practices for Internal 
Auditing" for a further discussion of modelling techniques. 
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° outline or procedural flowcharts or other pictorial methods of modelling 


systems and procedures, sometimes supplemented with decision tables. 


The level of detail necessary for any model of the entity is a matter for the auditor's 
judgment, but normally will be influenced by the nature and complexity of the 

audit entity and the scope of the audit in question. Additionally, the skills available 
within the audit team for preparation and interpretation of models may place further 
constraints on the detail or specificity to be provided. The auditor must establish 
whether, in fact, any modelling is necessary and establish whether the benefits 


outweigh the cost of preparation (time and resources). 


Documentation of the systems and processes of the audit entity most often will 


address the following areas: 


Organizational Structure and Relationships 


For most audits and particularly for functional and organizational audits, models of 
the management organization structure and relationships and their dynamics are 
valuable to the auditor. Documentation normally can take the form of organization 
charts which may be available from the audit entity or require preparation by the 


auditor. Depending on the nature of the audit the charts may outline: 


e the hierarchy of corporate management and any direct reporting 


relationships; or, 


* the dynamic organizational structure of corporate management 
encompassing both direct and functional relationships and respon- 
sibilities. 
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Management Control Bromevore ua 


The auditor can document the system of management controls by: 


e developing narrative system descriptions; 

@ preparing outline or detailed procedural flowcharts; 

@ identifying key decision and control points; and, 

® identifying procedures in place to control exceptional situations. 


The auditor will require an understanding of general accountability relationships 
and requirements. To obtain such an awareness, the auditor will review adherence 
to central agency policy directives, comformity with corporate policies and 
guidelines and management control over financial, human and physical resources 
and program performance. The review must be appropriately documented in either 


narrative or outline form. 


Whatever form the auditor's system description or accountability relationships 


take, they should always include as a minimum for each area examined: 


e the objectives of the system and its place in the organization; 

® a description of the activity, its influence and its results; 

@ key volume/dollar statistics; 

® organizational and management authorities and responsibilities; and, 

® identification of key controls and linkages to other control mechanisms. 


12 See Volume II, Part 2, Chapter 3, "Management Control: Concepts and 
Practices" for a detailed discussion. 
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Program and Support Activity Delivery Systems 


Preparation of program structure or logic models may be useful to enable the 
significant aspects of a program to be identified for discussion with auditee 


management. The auditor can outline program structure and logic by: 


e defining significant program components or activities including defining 


organizational units and functions established to provide for program 


delivery; 
8 determining the program/activity delivery system; 
& determining the outputs related to each program component; 
e assessing whether program objectives are consistent with the mandate 


of the entity or identifying where variances appear to exist; 


@ assessing measurement and reporting of program effectiveness; and 


e repeating the foregoing sequence for support activities. 


Validation of Data 


Having prepared documentation on the process or system under review the auditor 


must now undertake limited testing procedures to: 

® verify the accuracy of the data collected; and, 

® develop an initial assessment of the operation of management controls. 
Methods employed to verify the accuracy of the data collected will include: 

2 a discussion of organization models, flowcharts and other system 


documentation with auditee personnel involved in the process in order 


to establish the accuracy of the representation; and, 
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® the selection of a sample of the elements of a process or transactions 


within a system for a walkthrough. 


A walkthrough is a practical method of verifying the accuracy of systems 
documentation which provides a preliminary indication of the existence and 


effectiveness of process!? 


and results controls. However, the walkthrough does 

not constitute a detailed verification procedure nor does it involve the collection 

and examination of all pertinent audit evidence. Notwithstanding these qualifications, 
it is a useful mechanism to validate the accuracy of systems data and indicate 


whether the system or process functions as described. 


The validation procedures may well disclose some discrepancies. There are two 


possible reasons for this: 


& the system description, or expected results documentation, is inaccurate; 
or, 
@ 
& the description is accurate but the system does not exist or behave as 
described. 


If the auditor finds a discrepancy or any information that is inconsistent with the 
system description, reference to the original information sources is required before 
proceeding with any further data collection or testing. Under normal circumstances, 
any system breakdown would be noted at this point as information useful for a 
preliminary assessment. Further analysis will be performed during the evaluation 
phase. Only in exceptional circumstances, such as suspected fraud, should the 
matter be pursued further at this time and reported to the audit manager or auditee 
forthwith. 


bas i3 Here the term "process" is used to represent the general wherewithal to 
produce results. 
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In formulating a preliminary assessment of the operations of management controls, 


the auditor usually will want to consider: 


ry the adequacy of resources; 

» the consistency between objectives and results; 

e the existence and apparent effectiveness of information systems; and, 
€ any other concerns or areas of apparent significance. 


Preliminary assessments are usually based on overview analysis and will not be 
substantiated by quantitative measures or detailed validation at this stage. From 
the limited testing performed during the validation of system descriptions, the 


auditor will be in a position to formulate preliminary observations relative to: 


@ the adequacy of organizational structures, plans, objectives and systems 


and procedures (i.e. the auditee's infrastructure); 


e the economy, efficiency and effectiveness of the organization (i.e. the 


auditee's results); and, 


@ the existence and apparent adequacy of controls over resources and 


operations. 


These preliminary observations should be documented in the working papers, but 
only in the exceptional circumstances, outlined earlier, will they be reported to the 


audit manager or auditee at this stage. 
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Limited Control Review and Identification of Potentially Significant Audit Issues 


The auditor is now in possession of a body of information relative to the structure, 
management and operation of the entity under examination, together with a 
predetermined control model which indicates the control objectives of the entity. 

In evaluating the adequacy of the actual control framework, the auditor must match 
the existing controls with the predetermined control objectives. This will enable 
the auditor to identify: 


# existing controls which meet the requirements of at least one control 
objective, including controls which operate in the environment external 


to the audit entity but affect its operations; 


@ control objectives for which no existing controls have been found; this 
identifies a potential weakness in the control framework of the audit 


entity; and 


® existing controls for which no control objectives exist in the predeter- 
mined control framework; this could be an indication of either redundant 
and/or uneconomical control or an incomplete predetermined control 


model. 


When an existing control is identified, it should be posted to a Register of Essential 
Controls (Table De along with all the control objectives from the predetermined 
control model which are satisfied by the existing control. Control objectives which 
cannot be matched to any control of the actual control framework should be listed 
in a Summary of Control Weaknesses (Table 3)!4. These two documents together 
will form the starting point of the evaluation phase where the auditor will be 


required to assess the adequacy of the auditee's control framework. 


14 The Register of Essential Controls and Summary of Control Weaknesses could, 
of course, be combined into one comprehensive table, if so desired. 
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At this stage, the auditor will have conducted a limited control review (compliance 


testing)!? 


to significant audit issues. These issues will normally be identified by the major 


and should be in a position to determine those areas which may give rise 


differences between the predetermined control model and the documented control 


framework. Other significant audit issues may arise from: 


© areas of concern identified in previous audits and still considered relevant; 
e apparent ineffectiveness of controls (as opposed to non-existence) 

® potential areas of uneconomical or inefficient operations; 

@ perceived weaknesses in the entity's capability to measure program 


effectiveness; and 


© other specific management and operational characteristics when these 


are judged to be unusual. 


The limited control review activity in the preceding outline, although performed as 
part of the review phase, is actually an evaluation activity. This is consistent with 
the view of the evaluation activity presented in Figure 2, which shows the evaluation 
phase as overlapping all the other phases. Significant audit issues will be 


substantiated in the verification phase of the audit assignment through substantive 


testing. 16 


15. See Volume II, Part 2, Chapter 2, "Control: Concepts and Applications for 
Internal Auditors" for a discussion of compliance and substantive testing. 


16 Idem. 
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Preparation of a Plan for the Evaluation Phase 


The purpose of the evaluation phase is to analyze the adequacy of the existing 
management control framework against the predetermined model and to identify 
any significant weaknesses and deficiencies. As indicated, the evaluation process 
has already started with the limited control review and associated activity described 
in the previous discussion. Therefore, the evaluation plan in the following discussion 


deals only with the remaining evaluation activities. 


In preparing an evaluation phase plan the auditor will use as a basis: the products 
of the review phase - system documentation, predetermined control models, Register 
of Essential Controls, Summary of Control Weaknesses and a list of potentially 


significant audit areas. 


The form of the evaluation phase plan will vary according to the nature of the 


assignment, but the plan should, as a minimum, contain: 


@ a list of audit issues considered significant upon which the evaluation 


phase will concentrate; 


e an outline of the audit procedures to be employed incorporating: 


methods for substantiating controls weaknesses; 


- sources of reference for evaluation (e.g. flowcharts or interviews); 


. an indication of the necessity for further, more detailed, system 


documentation; 


- methods for performing cause-effect analyses; and 


- an indication of areas likely to require further verification and 


the possible approach to be adopted. 


e assignment of responsibility for detailed audit tasks; and, 


Internal Audit Handbook 
Volume II, Part 1 
Chapter 2, Section 2 - 52 - 


® identification of documentation to be employed for the evaluation. 


SECTION TWO: PRINCIPAL PRODUCTS 


The principal products of the review phase are: 


r) a current predetermined control model(s); 

e detailed documentation of existing management control framework; 
® a "Register of Essential Controls"; 

€ a "Summary of Control Weaknesses"; and 


® a list of potentially significant audit issues. 


LOWE-MARTIN 
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CHAPTER THREE 


EVALUATION PHASE 


INTRODUCTION 


Evaluation of the management control framework is the core of the audit process. 
It calls for the exercise by the auditor of a large measure of expert analysis and 
judgment. Evaluation, as an activity, is an iterative process and for this reason it 
cannot be regarded simply as part of a linear progression from the review phase to 
the verification phase. The evaluation activity begins during the planning of the 
assignment and continues through to the drafting of the final audit report. An 
understanding of these characteristics of evaluation has been assumed in the 


development of this guide. 
SECTION ONE: PRINCIPAL COMPONENTS 
The principal components of the evaluation phase are: 
® comparison of descriptive model vs. predetermined control model (11.5)! i, 


® preparation of control registers and determination of important controls 


(existing and missing) (II.6, II.7); 


® determination of requirement for additional data; and preparation and 


approval of detailed verification plan procedures (II.8, II.9); 


® analysis of causes and effects of audit observations (II.11); 

® development of audit findings, conclusions and recommendations (II.12); 
and, 

e completion and review of audit working papers (II.13). 


——— 


17. The numbers is parentheses refer to the components of the audit process as 
shown in Figure 3. 
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Evaluation of the Existing Management Control Framework against the 
Predetermined Model 


This activity is started informally in the planning phase and carried out mostly 
during the review phase, but not completed until after the verification phase (see 
Figure 5 for an overview of the control evaluation process), During the review 
phase the auditor has matched the predetermined control model to the actual 
control framework and identified both existing controls and potential control 
weaknesses. During the planning for the evaluation phase, the auditor will have 
identified which audit issues are considered significant and upon which the audit 
will now concentrate. In order to evaluate the adequacy of the control framework, 
the auditor must now identify the existing controls and apparent control weaknesses 
which are critical and which warrant further evaluation and possible verification 


before a decision can be made as to inclusion in the audit report. 


With respect to the critical objectives that the auditor has concluded are relevant, 
the evaluation process should be completed, as far as possible, by reference to the 
information gathered during previous phases of the audit. Assessment should be 

Supported by evidence and referenced to its source. If insufficient evidence exists 


at this point, it should be noted for inclusion in the verification plan. 
Identification, Evaluation and Documentation of Control Weaknesses 


The purpose of this critical component of the audit assignment is to review the 
Register of Essential Controls and Summary of Control Weaknesses prepared in the 


review phase and identify: 


8 management and process controls which are essential and critical to the 
effective operation of the system and are considered to have or 


suspected of having significant weaknesses; 


& controls which serve a useful role in the operation of the system but are 


not considered critical enough to warrant further investigation; 
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cS controls which appear to be of no purpose to the organization or to the 
environment surrounding the organizaton; that is, controls for which no 


matching control objectives exist in the predetermined control model; 


® control deficiencies or weaknesses likely to give rise to significant loss, 
error or inefficiency and for which further investigation is likely to be 


cost effective; and 


S control deficiencies and weaknesses not considered significant enough 


to warrant further investigation. 


The assessment by the auditor of the significance of control weaknesses should be 


recorded in the Summary of Control Weaknesses. This summary will provide: 


e a means of documenting in an organized manner the control weaknesses 


deemed significant; 


® a record of procedures and tests developed to test the effectiveness of 


essential controls or to assess the effect of control weaknesses; 


® a record of the results of the substantive testing of essential controls; 
and 
e a record of weaknesses in the operation or absence of essential controls. 


At this point in the audit, the auditor may wish to report, on an informal basis, to 
the auditee any information on control weaknesses which are deemed to be not 
significant, but for which minor problems have been identified which could be easily 
rectified by the auditee management. This will help ensure continuous and effective 
communication between the auditee and the auditor and will enhance the perception 


of the audit function as a helpful tool to management at all levels. 
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Development of a Detailed Verification Plan 


The auditor must now determine whether additional verification is required to 
substantiate findings, hypotheses and assumptions as illustrated in Figure 5 and if 
so, the extent of the verification procedures required. This will require consideration 


of: 


e those issues identified earlier in the audit which now appear less 


Significant and warrant no further investigation; 


@ those issues which are of major significance but require no further 


verification to support audit observations; 


® those issues of major significance which require further verification; 
and, 
® the essential controls identified as being in place, the existence and 


effectiveness of which must be substantiated. 
The objectives of the verification plan are: 


% to determine that controls listed in the summary of essential controls 


operate as designed and are effective; and, 

6 to gather data in support of the analysis of the cause and effect of the 
significant weaknesses, deficiencies and inefficiencies listed on the 
Summary of Control Weaknesses. 


To achieve the verification audit objectives the verification plan should be designed: 


% to provide assurance that the balance of systems are operating 


effectively; 


6 to verify deficiencies, inefficiencies and weaknesses; 
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e to determine or verify assumptions as to the causes and effects of 


identified weaknesses; 


* to outline the detailed objectives and procedures for each verification 
step; and, 
& to employ techniques appropriate to the objectives and the nature of 


the audit unit being examined. 


Given that verification activity is typically a costly one, it should be noted that if 
the auditor proposes any major changes to the verification phase from that planned 
for during the assignment planning phase, it may be appropriate for such changes 
to be reviewed and approved by the audit manager prior to commencement of 
detailed verification procedures. The purpose of this review is to eliminate the 


possibility of any misdirected audit emphasis. 


Where warranted, the auditor would now proceed with the verification phase of the 


audit (see Figures 5 and 6). 
Analysis of Causes and Effects of Audit Observations 


On completion of the necessary procedures to substantiate existing controls and 
significant deficiencies, inefficiencies and weaknesses, the auditor will be in 
possession of a body of audit observations recorded on the Register of Essential 


Controls and the Summary of Control Weaknesses. 


To develop the audit findings, conclusions and recommendations, the causes and 
effects of audit findings must be analyzed as depicted in Figure 7. The focus of 
this analysis is to substantiate hypotheses on the reasons for, and the significance 
of, failure to match the specified criteria. The amount of analysis should be cost- 


effective. 
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DEVELOPMENT OF VERIFICATION PLAN 


REGISTER OF 


VALIDATED INTERNAL 
SYSTEMS CONTROLS 
DOCUMEN- 


TATION 


REGISTER OF 
EXTERNAL 
CONTROLS 


SUMMARY 


OF CONTROL 
PREDETER- 
MINED WEAKNESSES 
CONTROL 


MODEL 


IIl.1 EXPANSION OF AUDITOR’S UNDERSTANDING OF ENTITY 

II1.2 DOCUMENTATION OF PROCESSES AND SYSTEMS (DESCRIPTIVE MODEL) 

11.3 VALIDATION OF DATA 

11.4 DEVELOPMENT AND/OR UPDATING OF PREDETERMINED CONTROL MODEL 

11.5 COMPARISON OF DESCRIPTIVE MODEL VS. PREDETERMINED CONTROL MODEL 
11.6 PREPARATION OF CONTROL REGISTERS 

11.7. DETERMINATION OF IMPORTANT CONTROLS (EXISTING AND MISSING) 

1.8 DETERMINATION OF REQUIREMENT FOR ADDITIONAL DATA 


Figure 6 
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CAUSE-EFFECT ANALYSIS 
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In the analysis of audit observations the auditor must bear in mind that: 
® cause and effect are inter-related; and, 


® causes may be external to the organization or process and may arise 
from the directives of central agencies or other external influences 


beyond the boundaries of the audit entity. 


The auditor should clearly state the problem which is revealed by an audit finding 
or group of findings. The findings should relate to the control weaknesses that 
were observed in the course of the review phase and substantiated in the verification 


phase. 


Having determined the problem and its cause the auditor must consider the possible 
effects on the audit entity. These will usually be derived from the control objectives 
outlined in the predetermined control model and backed by appropriate cost-benefit 


analysis. Figure 8 provides examples for the schematic form shown in Figure 7. 
Development of Audit Findings, Conclusions and Recommendations 


At this stage of the assignment, the auditor will have accumulated a list of findings 
from the review and evaluation phases of the audit. Several will have been 

substantiated further by various verification procedures and analyzed for cause and 
effect. For each finding a summary should be prepared which will normally outline 


the following: 


e area/activity audited; 
® summary of the audit findings, quantified where possible; 
® analysis of causes and effects; and, 


® reference to detailed audit working papers. 
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EXAMPLE OF FINDINGS - CAUSE AND EFFECT ANALYSIS 


AUDIT FINDINGS 


THE ORGANIZATION IS INEFFECTIVE IN 
THE DELIVERY OF ITS 
PRODUCTS/SERVICES 


RESOURCES ARE NOT USED 
OPTIMALLY: RESOURCES ARE LOST 


DATA ARE NOT REPRESENTATIVE 


COMMUNICATIONS/POLICIES ARE NOT 
FOLLOWED 


PRODUCT DEMANDS ARE POORLY 
PROCESSED OR ARE NOT PROCESSED 


POSSIBLE EFFECTS 


1. CUSTOMER DISSATISFACTION; 

ORGANIZATIONAL BANKRUPTCY. 

2. INCREASED PRODUCT/ADMINISTRATIVE 
COSTS (E.G. $X INCREASE IN COST/UNIT 
PRODUCED); ORGANIZATIONAL INEFFECTIVENESS 
3. FAULTY MANAGEMENT DECISIONS (E.G. $X MILLION 

INVESTMENT ACCEPTED; SYMILLION LOSS 

RESULTED LAST YEAR; $Z MILLION CONTINUING 

LOSS IF INVESTMENT NOT TERMINATED 

IMMEDIATELY); ORGANIZATIONAL INEFFICIENCY 

AND INEFFECTIVENESS. 

4. INCREASED ERROR RATES, ORGANIZATIONAL 

INEFFICIENCY AND INEFFECTIVENESS. 

5. DECREASED PRODUCT DEMANDS (E.G. $x 
MILLION DROP FROM PREVIOUS YEAR); 
ORGANIZATIONAL INEFFECTIVENESS (E.G. IN 
PRIVATE SECTOR: PROFITABILITY REDUCED 

FROM X% TO Y% OR, IN PUBLIC SECTOR: 
X APPLICANTS DID NOT RECEIVE 

PASSPORTS THAT THEY 
APPLIED FOR). 


POSSIBLE CAUSES 


1. UNAVAILABILITY OF RESOURCES; 
POOR DISTRIBUTION SYSTEM 

2. UNREPRESENTATIVE DATA; 
ORGANIZATIONAL INEFFICIENCY; 
INADEQUATE PHYSICAL CONTROLS 

3. INADEQUATE INFORMATION SYSTEMS 
CONTROLS 

4. INADEQUATE SYSTEMS CONTROLS 

5. UNAVAILABILITY OR 

INAPPROPRIATENESS OF RESOURCES; 

INADEQUATE PRODUCTION 

CONTROLS 


Figure 8 
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From this summary the auditor will be able to refine the findings into preliminary 


conclusions for inclusion in the draft audit report. 
There are two types of audit findings - opinion-based and fact-based. 


Opinion-based findings may relate to such issues as:. 


8 quality of decision processes; 

e adequacy of work methods; 

e coherence of systems; 

® integrity of communication and information systems; and, 
® the need for control procedures. 


Fact-based findings are those which may be expressed in terms of factual data - 


dollars, units of production, person years, etc. 


In developing audit findings the auditor should ensure they are: 


® material or significant in relation to the unit examined; 

® supported adequately and appropriately by audit evidence; 
@ validated and agreed with auditee management; and, 

@ analyzed with respect to cause and effect. 


In the development of audit conclusions, the effect or implication of the deficiencies, 


inefficiencies or weaknesses identified in the findings will be recorded. 
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The auditor will normally develop conclusions from: 


o an evaluation of variances resulting from deficiencies or weaknesses; 
® measured effects of those weaknesses; and, 
® discussion of results with auditee management. 


Audit conclusions should be: 


* based on a careful analysis of the effects of problems identified in the 
audit unit; 

e stated clearly and unambiguously; 

& expressed in quantifiable terms whenever possible; and, 

® developed with an awareness of the total environment within which the 


unit operates. 


The development of recommendations should offer reasonable and practical 
suggestions for corrective action to remedy control deficiencies, inefficiencies and 


weaknesses, 


The auditor should consider the possible and feasible corrective actions available in 
the context of the audit unit and must assess carefully the cost-benefit 
considerations of recommending additional controls. Once a range of feasible 
suggestions has been developed, the auditor must select the most appropriate one 
and express, as clearly as possible, the recommended course of action and the 
responsibility for implementing the recommendation. Audit recommendations 
should always be so framed as to address what course of action is suggested, but 
should not venture into the manner of implementation, (the how) as this is the 


domain of management. 


Figure 9 illustrates the process of developing audit findings, conclusions and 


recommendations. 
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DEVELOPMENT OF AUDIT FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 


PROCESS 


. CONSIDER SIGNIFICANCE/ 

RELEVANCE 

2. DETERMINE FACT-BASED/ 
OPINION-BASED FINDINGS 

3. ENSURE ADEQUATELY 
SUPPORTED AND SUFFICIENT 

4. ENSURE FACTS AGREED WITH 
AUDITEE 

5. EXPRESS CLEARLY AND 

CONCISELY 


AUDIT 
EVIDENCE 


PROCESS 


1. QUANTIFY EFFECT/ 
PREDICT IMPLICATION 

2. CONSIDER ROOT 
CAUSES 

3. EXPRESS CLEARLY AND 

CONCISELY 


ANALYSIS OF 
CAUSES AND 
EFFECTS 


CONCLUSIONS 


PROCESS 


. CONSIDER STATE OF THE 
ART 

. CONSIDER CONSTRAINTS 

. CONSIDER OVERALL 
ENVIRONMENT 

4. ENSURE FEASIBLE AND 

COST BENEFICIAL 

5. EXPRESS CLEARLY ACTION 

RECOMMENDED AND 

RESPONSIBILITY THEREFOR 


POSSIBLE 
RESOLUTIONS 


RECOMMENDATIONS 


Figure 9 
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Completion and Review of Audit Working Papers 
Audit working papers should be reviewed on a regular basis by the team leader as 
the audit progresses. As a minimum, the audit manager should review them at the 


end of the verification phase as illustrated in Figure 10. 


The audit manager's review should ensure that: 


% audit coverage has been adequate; 
° files and working papers are complete; and, 
e appropriate evidence has been obtained and documented to support 


audit observations. 


To adequately substantiate audit findings the working papers should include: 


% audit procedures and the nature and extent of audit work performed; 
® appropriate documentation of unit's systems, activities and controls; 
s evidence of supervisory review; 

+ evidence of a quality control review, where applicable, of the conduct 


of the audit; and, 


a appropriate index and cross-references. 


Standard review checklists should be developed to provide the basis for, and evidence 
of, supervisory review. Appropriate levels of directorate and project management 
will complete applicable portions of the review checklist in the course of their 
review, and the completed checklist will form part of the audit working papers. 

(See Volume I, Chapter 4 for a further discussion of working paper review and sample 


check lists). 
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WORKING PAPER REVIEW 


REGISTER OF 
INTERNAL 
CONTROLS 


SUBSTAN- 
TIATED 
CONTROLS 


VERIFICATION 
PHASE 
PLAN 


REGISTER OF 
EXTERNAL 
CONTROLS 


VERIFIED 
WEAKNESSES 


SUMMARY 
OF CONTROL 
WEAKNESSES 


FINDINGS 
CONCLUSIONS 
RECOMMEN- 
DATIONS 


AUDIT 
WORKING 
PAPERS 


AUDIT 
REPORT 


11.7 DETERMINATION OF IMPORTANT CONTROLS (EXISTING AND MISSING) 

11.8 DETERMINATION OF REQUIREMENT FOR ADDITIONAL DATA 

11.9 PREPARATION AND APPROVAL OF DETAILED VERIFICATION PROCEDURES 

11.10 PERFORMANCE OF DETAILED AUDIT TESTS AND OTHER VERIFICATION PROCEDURES 
11.11 ANALYSIS OF CAUSES AND EFFECTS OF AUDIT OBSERVATIONS 

11.12 DEVELOPMENT OF AUDIT FINDINGS, CONCLUSIONS AND RECOMMENDATIONS 

11.13 COMPLETION AND REVIEW OF AUDIT WORKING PAPERS 


HiI.1 PRESENTATION OF AUDIT 
FINDINGS TO AUDITEE 
111.2. PREPARATION OF AUDIT 
REPORT 
IfI.3. PRESENTATION OF AUDIT FINDINGS TO AUDIT COMMITTEE 
IfI.4 AUDIT REPORT FOLLOW-UP 


Figure 10 
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SECTION TWO: PRINCIPAL PRODUCTS 


The principal products of the evaluation phase are the following: 


@ Register of Essential Controls; 

a Summary of Control Weaknesses; 

® plan for the verification phase; 

% audit findings, conclusions and recommendations; 


& completed audit working papers. 
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CHAPTER FOUR 


VERIFICATION PHASE 


INTRODUCTION 


This phase of the audit concentrates on the in-depth verification of essential controls 
and the further substantiation of significant deficiencies, inefficiencies and 
weaknesses, It is designed to collect sufficient additional, reliable evidence to 
Support audit findings and to report on significant issues. It is essentially an adjunct 
to the evaluation phase, designed to consolidate preliminary observations and to 
provide evidence for the analysis of causes and effects and development of audit 


findings which are the principal output of that phase. 


SECTION ONE: PRINCIPAL COMPONENTS 


The principal components of the verification phase are: 


e preparation and approval of detailed verification procedures (u1.9)48; 
and, 
@ performance of detailed audit tests and other verification procedures 


(11.10). 


Preparation and Approval of Detailed Verification Procedures 


It is the responsibility of the team leader to prepare the detailed audit procedures 
for use in the verification phase. Once the procedures have been developed, the 
team leader should review them with the audit manager before field testing 
commences. When the audit manager has approved the procedures as appropriate 


for use in the audit, the team leader assigns responsibilities for their execution. 


18 | The numbers in parentheses refer to the components of the audit process as 
shown in Figure 3. 
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Some of the more important steps involved in the development of audit procedures 


for verification testing normally will include: 


Reviewing Specific Audit Criteria 


Specific general and detailed audit criteria will have been developed during the 
review phase, but may require further review and development at this stage. Detailed 
procedural testing of controls may require more detailed sub-criteria and these 

should be developed and discussed with the auditee prior to conducting verification 


testing. 


Defining the Population 


The population to be tested will depend on the objectives of the audit. In an audit 
of a transaction processing system the population will be all transactions taking 
place during the period under review. In an audit of effectiveness the population 
may be more restricted or significantly expanded. Whatever the size of the 
population, the auditor should attempt to determine its principal characteristics 


and the best means for reviewing them. 


Selection of Methods of Testing 


Once the auditor has established the principal features of the population, the most 
appropriate method of testing must be determined. Some techniques are discussed 


in Part 1, Chapter 2 and a more detailed discussion is contained in Part 2, Chapter 4. 
Performance of Detailed Audit Tests and other Verification Procedures 


Once the audit procedures have been developed and approved by the audit manager, 


the detailed field testing will begin. The testing will focus on: 


° substantiation of the existence and effective operation of the controls 


listed on the Register of Essential Controls; and, 
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) verification of the significant weaknesses and deficiencies listed on the 
Summary of Control Weaknesses during the review and evaluation 


phases, 


Verification normally involves the application of given audit procedures to a selected 
group of transactions/activities and is normally performed on a sample basis. The 
nature and extent of audit evidence required, which determines the level of testing 
to be employed, will itself depend on the level of materiality or significance of the 


issue under review and the level of assurance required to support audit findings. 


Audit evidence is required to support audit conclusions and findings. What constitutes 
audit evidence will be determined in general terms by the nature of the audit 
engagement. For example, in function and organizational audit engagements, the 
evidence gathered must support, among other things, findings regarding the manner 

in which management responsibilities have been fulfilled. In this latter situation, 
audit evidence will often be persuasive rather than conclusive, and may be derived 


from interview and observation rather than by transaction testing and analysis. 


Accordingly, testing methods will vary depending on: 


cy the nature of the activity being audited; 
Y the purpose of the test; and, 
e the type of evidence available. 


The auditor should always attempt to select testing methods which are appropriate 
to the environment, cost-effective and designed to give the level of coverage and 


assurance necessary to support audit findings. 


A wide range of verification techniques is available to the auditor. Some of their 


principal features and applications are outlined as follows. 
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Sampling 


When a population to be tested is too large to test 100 per cent in a cost-effective 


manner then some form of sampling will be employed. 


Two broad categories of sampling are available: Non-statistical (often referred to 
as "Judgmental") and statistical. A detailed discussion of the advantages and 
disadvantages of each is beyond the scope of Part | and, in any case, this subject is 


well covered in a number of textbooks (see Bibliography for examples). 


Whether statistical or non-statistical sampling is used, any representative selection 


will involve the following steps: 


definition of objectives; 
population definition; 

choice of degree of assurance; 
determination of sample extent; 
selection of sample items; 
verification of sample items; 


evaluation of sample results; 


conclusion. 


Statistical Sampling 


Statistical sampling is a rigorous tool for the systematic collection and mathematical 
evaluation of data. It involves the inspection and analysis of a portion of a popula- 
tion. Application of the laws of probability provides a predictable level of precision 
to, and confidence in, the sample information which can be related to (i.e. is 
representative of) the entire population. Statistical sampling is of particular value 
in arriving at a conclusion on the characteristics of large, generally homogeneous 


populations. The most widely used sampling applications are: 
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* attribute sampling, which answers the question of "how many?" and is 
used to determine the characteristics of a population in numerical terms; 


and, 


& variable sampling, which answers the question "how much?" and is used 
to obtain the estimated monetary value of a population from a sample 


of that population. 


Computer Assisted Audit Techniques 


There are three major techniques which will be of use to the auditor: 


® test packs, by which the auditor can ascertain whether the controls 


residing in the hardware and in the program are operating correctly; 


e computer audit programs, which are written by, or for, the auditor to 


test the integrity of information produced by a particular system; and, 
a continuous monitoring (sampling) hardware or software. 


These applications require a good deal of expertise and considerable developmental 


effort where standard packs, or sampling software, are not available. 


Analytical Review and Variance Analysis 


Analytical review is the systematic analysis and comparison of related figures, 
trends and ratios used to identify their mutual consistency or inconsistency. 
Analytical review requires an inquiring mind and a good working knowledge of the 
audit entity. These will then be applied to financial or operating statements to 
determine the existence of logical relationships between known facts, estimates 

and budgets, corresponding data for previous years and the development of trends 

or variances. If unusual or unexpected trends, relationships or variances are 
identified, the auditor then will require plausible explanations or supporting evidence. 
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Independent or Third-party Confirmation 


This technique is generally applicable to financial audits where external evidence is 
sought by the auditor to support account balances. However, it may have other 
applications to confirm the status of agreements, legal position under contracts or 


litigation, etc. 


Other Techniques 


A number of other specialized techniques may be considered by the auditor in 


particular situations and may include: 


correlation/regression analysis; 
time and motion study; 


linear programming; 


network analysis. 

Whatever verification methods are used by the auditor, they must be designed to 
provide adequate and appropriate audit evidence relative to the operation of essential 
controls and the existence of significant control deficiencies and weaknesses and 
their cause and effect. (See Volume II, Part 2, Chapters 4 and 5 for further discussion 
of analysis techniques and audit evidence respectively.) 

SECTION TWO: PRINCIPAL PRODUCTS 

The principal products of the verification phase are: 


e substantiation of essential controls (1.10)! 7; 


a verification of significant control deficiencies, inefficiencies and 


weaknesses (II.10); and, 


° verification of causes of deficiencies, inefficiencies and weaknesses, 


and of their effects on the auditees operations (II.10). 


19 The numbers in parentheses refer to the components of the audit process as 
shown in Figure 3. 
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CHAPTER FIVE 
REPORTING PHASE 


INTRODUCTION 


Reporting is the mechanism by which the auditor communicates findings, conclusions 
and recommendations to management to assist them in monitoring the economy, 
efficiency and effectiveness of internal management practices and controls, in 
improving the control framework and in ensuring adherence to established policies, 
plans and procedures. It is essential that the auditor develop the report for the 
appropriate audience. In certain situations, this may require various levels of 


reporting to be developed. 
SECTION ONE: PRINCIPAL COMPONENTS 
The principal components of the reporting phase are: 
® presentation of audit findings to the auditee (ilsi)-e 
® preparation of the audit report (III.2); 
@ presentation of audit findings to the Audit Committee (III.3); and, 
8 audit report follow-up (III.4). 
Presentation of Audit Findings to the Auditee 
There should be continuous dialogue between auditors and auditee management 
throughout the course of the audit assignment. There are, however, certain formal 
communication requirements, one of which is the requirement for the team leader 


to provide auditee management with a summary of audit findings and recommendations 


prior to drafting the final report. Responsibility for arranging and conducting such 


20 The numbers in parentheses refer to the components of the audit phase as 
shown in Figure 3. 
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a debriefing rests with the team leader. In certain circumstances, it may be 
appropriate for the audit manager or even the director to be present at these 


debriefings. 


The purposes of the exit interview are the following: 


8 to provide the auditor with an opportunity to ensure that all relevant 
facts related to the findings were considered. Accordingly, the auditor 
could obtain additional information or insight at this stage which was 
unavailable earlier and which could have a significant effect on the 
presentation and subsequent acceptability of the audit findings; also, 
as a result of any new information, there may be a need for further 


field work on specific items; 


8 to establish a firm base on which to write the audit report; after the 
exit review there may be disagreement on conclusions and recommenda- 
tions, but there should be no disagreement or subsequent surprises on 


factual matters; 


& to ensure that auditee managers have a thorough knowledge and under- 
standing of the audit findings and of the conclusions and recommenda- 


tions proposed by the auditor; 


6 to give auditee managers an opportunity to present information and 
opinions which may influence the auditor's conclusions, and allow the 
auditee managers the opportunity to suggest their own recommendations 
on how suggested improvements can be implemented and inform the 
auditor of action already taken; also, this process will afford auditee 
managers the opportunity to initiate corrective action immediately 


rather than waiting for the formal audit report; and, 


e to provide a courteous conclusion to the audit and give the team leader 
a further opportunity to adjust, as necessary, the tone of the audit 
report; also, it can demonstrate to the auditee that the auditor is acting 


in a cooperative manner, rather than as an autonomous critic. 
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Once the team leader is satisfied that all working papers have been completed and 
reviewed and a summary of audit findings, conclusions, recommendations prepared, 
a meeting with the appropriate auditee management should be arranged before 
leaving the audit site. The team leader should address, as a minimum, the following 


matters during the exit interview: 


® the purpose, scope and objectives of the audit; 

% all significant audit findings, conclusions and recommendations to be 
incorporated in the report; 

® acknowledgement of any difficulties or constraints experienced by the 
auditor in conducting the review; 

& comments on matters already corrected; and, 

® the cooperation extended during the audit. 


If the discussions are conducted in a positive spirit, it is likely that auditee 
management will be disposed to act quickly to remedy observed deficiencies. 
Management views should be solicited and remedial action noted to ensure that the 


final audit report becomes a constructive mechanism for action. 


Under certain circumstances, it may be appropriate to incorporate a visual 
presentation to senior levels of management as part of the exit debriefing. Such a 
presentation normally will be oral with various visual aids. Careful planning should 
precede such a presentation to ensure that audit findings which are appropriate to 


the level being debriefed are communicated logically, clearly and concisely. 


It is a prerequisite of all exit meetings, debriefings or presentations that the team 


leader be clear as to the substance and tone of the final report. 
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Preparation of the Audit Report 


The purpose of the audit report is to supply auditee managers with timely, accurate, 


concise and relevant information needed to initiate corrective actions. 


Responsibility for preparation of the draft audit report should be shared by all 
persons engaged on the audit, with the various sections or elements of the report 
being developed by those auditors involved during the fieldwork, in the manner 
described in the evaluation phase. However, the team leader has overall 
responsibility for the preparation of the final report and for ensuring the quality 


and clarity of the final product. 


There is no standard or universal model for the audit report. However, all reports 


should incorporate the following elements: 


e an outline of the audit purpose, scope and objectives; 
€ a brief description of the audit approach followed; 
@ an overview of the audit entity in sufficient detail to convey that the 


auditor has gained a good general perception of the nature of the entity, 


and the environment in which it operates; 


e a compendium of the major findings, conclusions and recommendations 
which states clearly the nature of the problem, the causes and the 
implications, and provides practical recommendations for action to be 


taken by each level of management, as necessary. 
& responses from management to each of the findings; and, 
® an executive summary. 
In preparing the report, the team leader must ensure that the audit findings are 
adequately supported by evidence contained in the working papers. It is advisable 


that a working copy of the draft report be cross-referenced to the working papers 


to facilitate reference to the supporting evidence. 
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Finally, in preparing the report, the team leader must exercise judgment in the 
selection of issues to be covered. The report should include only relevant and 
significant information expressed logically and concisely, and addressed clearly to 
those who have the responsibility and authority for initiating corrective action. To 
accomplish this it may be necessary to provide more than one level of summarization 
and to supply exerpts to relevant peer managers (e.g. chief financial officer, 
personnel manager). The auditor's credibility can be seriously reduced by the 
inclusion of irrelevant or immaterial findings, or through a lack of clarity in assigning 
responsibility for the implementation of action plans; minor findings can always be 


conveyed to the auditee orally or in a management letter. 
Presentation of Audit Findings to the Audit Committee 


Arrangements for presenting audit results to Audit Committees vary considerably 
from organization to organization. In some cases each audit report is presented as 
it is completed; this allows the Audit Committee to participate in the disposition 
of recommendations, intervening, if considered necessary, at the action plan 
development stage, when such action is most effective and efficient. However, 
this mode of operation implies a frequency of Audit Committee meetings which 


some organizations may find too time consuming. 


Audit Committees in other organizations are satisfied with a periodic summary of 
major findings, sometimes as infrequently as once a year in the internal audit head's 
Annual Report. This mode is less time consuming, but also less open to timely 
intervention in action plan decisions and monitoring of implementation. Also, this 
mode of operation obviously precludes debriefing of individual reports to the Audit 


Committee. 


The recommended mode of operation is that frequency of Audit Committee meetings, 
and corresponding degree and nature of intervention, should be determined by the 


perceived need for such intervention. 
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Audit Report Follow-up 


Having presented the report to auditee management the auditor has communicated 

to management deficiencies which, in the auditor's judgment, will expose management 
to material risks unless they are corrected. The prime responsibility for implemen- 
tation of the corrective actions lies with the auditee management and not with the 
auditor. However, the auditor does have a responsibility to report on the progress 


of implementation of audit findings. 


Once the audit report has been issued, the senior managers responsible for the unit 
audited should provide the internal audit group with a copy of the action plan 
developed for the implementation of recommendations contained in the report. 
This plan should contain implementation dates and should be submitted within a 
reasonable time following issuance of the audit report. (This requirement is often 


stipulated in the departmental Policy and Procedures Manual.) 


Once the action plan has been received, the head of internal audit will normally 
assign an auditor the responsibility for follow-up on the implementation of auditee 


action plans. The follow-up will normally consist of: 


® review of action plan; 
® advice to the auditee on the suitability of the plan; 
e advice to the deputy head and/or audit committee on the suitability of 


the plan, as appropriate; and 


) periodic review of implementation progress (as directed by internal 


audit policy and/or the audit committee). 


During the course of implementation, the auditor should monitor progress reports 
to the degree required by the deputy head (as advised by the audit committee). 
The auditor and audit manager will determine whether implementation of the action 


plan has been satisfactory, and where appropriate, report inadequate action. 
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Occasionally, the auditor may consider that a separate follow-up audit is warranted. 
In this situation, a detailed memorandum outlining the circumstances is prepared 

for the audit manager so that additional work may be included in the long-term 

plan. The audit may be included in the regular cycle or performed earlier, depending 
on the circumstances and, possibly, advice from the audit committee. The scope 
may consist of either the full scope of the audit or a scope limited to those areas 


with significant deficiencies. 


As a minimum, the auditor should include corrective action as part of the scope of 


any future audits in the area. 


(See Policy Interpretation Notice (PIN) 1983-03 for a more detailed treatment of 
Audit Follow-up.) 


SECTION TWO: PRINCIPAL PRODUCTS 


The principal products of the reporting phase are: 


® an oral debriefing/presentation to the auditee; 

a an audit report; 

@ a presentation to the Audit Committee; 

® feedback to auditee managers and/or the Audit Committee on the 


adequacy of the action plan and implementation progress. 
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CONCLUSION 


The audit process is a complex of concepts, procedures and relationships which is 
rarely linear in its form, logic or application. In conducting an internal audit 
assignment, the auditor is commonly faced with unfamiliar situations, several levels 
of interpersonal and organizational relationships and frequently with severe 
constraints on available resources and information availability. What this demands 
from the team leader, above all else, is the application of expert judgment, based 


upon wide and varied experience. 


It is to this audience that the guidance provided in Part 1 is addressed and 
accordingly, is not intended to be exhaustive or prescriptive. It is a means of 
providing a framework within which users will evolve their own specific strategies 
for achieving their objectives and, consequently, it seeks to deal with the principal 


features of conducting internal audit assignments in a wide range of environments. 


To achieve its objective of providing a reference point to auditors responsible for 
various types of internal audit assignments, Part 1 covers the major areas of the 
internal audit process in a generic manner and occasionally illustrates general themes 
with specific examples. Specific audit concepts and techniques are presented in 

Part 2 of Volume II of this Handbook. 


It is not the intention that the Handbook serve audit team leaders as a procedural 
handbook for conducting audit engagements. Rather, its objectives will be achieved 
if it communicates clearly the complex and iterative nature of the audit process, 

the necessity of communication and collaboration between auditor and auditee, and 
the crucial importance of exercising professional judgment at all times. These 
themes, which recur throughout, are central to the successful conduct of any internal 
audit assignment. With a keen awareness of them, it is hoped that audit team leaders 
will find in the detail of Part | a useful point of reference in applying the audit 


process to specific assignments. 
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Appendix A 


ILLUSTRATION ! OF RISK ANALYSIS 
FOR ASSIGNMENT PLANNING PURPOSES 


Assume that we are planning an audit of a program delivery responsibility centre 
which has a manual program delivery process and is contemplating the possibility 
of automation. 


According to the management information system (see Table 1) the cost per unit of 
output has been going up over the years, generally in proportion to inflation 
(essentially wage rates), however, we have also noted that in recent months the 
range of variability of the mean cost per unit has been increasing, while the upward 
trend in cost per unit has reversed itself (i.e. the most recent mean is lower than 


that justified by inflation). 


The increases in variability and the trend reversal in the mean cost per unit have 


been discussed with management and the following relevant data were elicited: 
f. The employees are aware that similar processes in the private sector 
are being automated and are, naturally, unsettled; morale is probably 
deteriorating. 


2. Turn-over in staff has been increasing in recent months. 


3. There has been a deterioration recently in the quality of time reporting. 


| The illustration presented here is a relatively complex example of risk 
assessment, which is justified only in equally complex resource allocation 
situations. An equivalent, purely judgmental, approach could be used in less 
complex situations. The judgmental approach is described in Volume I, 
Chapter 3 of the Internal Audit Handbook as well. 
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Table 1 


Cost/Unit Data for the Previous 


Five Years and for Recent Months 


(a) Cost/unit in years 1980-1984 


Year Cost/Unit 
1980 $100 
1981 $110 
1982 $125 
1983 $130 
1984 $135 


(b) Cost/Unit in 1985 


Month Cost/Unit 
January $130 
February $125 
March S132 
April $120 
May $128 
June $121 
July S19 


Mean cost/unit $875 + 7 = $125/unit 
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4, There seems to be no readily identifiable evidence of improvement in 
productivity; in fact, the increased turn-over should normally have 


decreased productivity. 


5. Management is concerned that the most recent management information 
systems (MIS) data may be unreliable for purposes of decision-making as 


to whether automation would be cost-effective. 


Management has estimated that the cost per unit obtainable through an automated 
program delivery system equivalent in capacity and producing the same quality of 
product as the present manual system would be $130/unit; current volume is 


$100 million units per year. 


It may be seen from an analysis of the foregoing that it is crucial for management 


to validate their current MIS information before taking a decision to automate. 


If their five-year trend is extrapolated, their cost/unit should be something over 
$135/unit, in which case automation would produce an annual operating saving of 

$5 million. This saving is clearly sufficient to recover the cost of the initial 
investment and provide an ample margin. However, if the productivity of the program 
delivery unit has indeed improved, perhaps due to fear driven increase in production 
per person, per hour, then investment in automation will produce a loss of $5 million 


per annum plus the investment cost. 


Given the high probability that the most recent cost/unit figures are not 
representative, it is likely that the manager would be willing to endorse a consider- 


able expenditure on consultant or audit resources in order to remove the uncertainty. 


The amount that the manager (organization) would be willing to spend would no 
doubt be somewhat less than the potential loss due to a wrong decision, but still 
substantial. In any case, in terms of allocation of scarce audit resources, if the 
audit manager were to perform similar analyses on other auditee areas competing 
for available audit resources, the final decision on audit resource allocation both 


between and within audit assignments would be greatly facilitated. 
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Appendix B 


ILLUSTRATION OF INDEXING/CROSS-REFERENCING 
SYSTEM FOR WORKING PAPERS 


CURRENT WORKING PAPER FILES 


(1) * (2) 


Subject Index References to References from 


General/Administration (File 1) 


Final Report; Final management letter A - C, D, G 
Management comments and action plans B A source (3), C 
Follow-up notes (S B source 

Draft report; Draft management letter D A E,G 

Verbal debriefing - notes E D source 
Supervisory review checklists and notes i (4) (4) 


Summary of audit observations, conclusions 


and recommendations (including cause/ 


effect analysis) G A, D Oe iy ic 
Assignment Planning Memorandum H (5) (5) 
Correspondence I as necessary source 


*For explanation of numbers see Explanatory Comments beginning on next page. 
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(5) (1) 


Subject Index References to References from 


Supporting Working Papers (File 2) 


Register of Essential Controls J G N, Q 
Register of Compensating Controls K G N, Q 
Summary of Control Weaknesses L G N,Q 
Control Questionnaires M N O 


Documentation in response to control 


questionnaires . N P if conclusive - 
a) aa 
if inconclusive - 
Q 
Predetermined control model O M source 


Documentation of existing control 


framework Pp N source 


Verification plan, procedures, results Q UF a a N, source 


Explanatory Comments 


(1) Expansion of index - Each page of the section can be referenced 
simply as Al, A2, A3 etc., if a worksheet is 
to be added between A2 and A3, then A2 
becomes A2.1 and the added sheet can be 
indexed A2.2. 
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(2) References to 


References from 


290 


Information contained in this section supports 
or provides background for the content of 
the section to which the reviewer is being 


referred. 


Information contained in this section is 
supported by the content of the section 


from which the reviewer is being referred. 


For ease of review, working papers should be cross-referenced in a manner 


reflecting the relationships between sections as noted above. Working papers 


should build upwards from source data to the final audit report. 


(3) Source 


(4) Supervisory review 
checklists 


(5) Assignment planning 
memorandum 


Indicates that information is derived directly 
from the audited entity through discussion, 


observation or application of audit procedures, 


Normally checklist is signed off indicating 
that a satisfactory standard of quality has 
been achieved in the audit and adequately 


reflected in the working papers. 


Various aspects of the memo may be referenced 
to supporting working papers indicating to 
the reviewer that planning decisions and 

scope were taken into consideration during 


the conduct of the assignment. 
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PERMANENT WORKING PAPER FILES 


Subject Index 
General Information AA 


Significant legal, financial and regulatory constraints that have an 


impact on the audited entity. 


Environmental information on organizations that affect the 


operations of the audited entity. 


Operational Base for Audited Entity BB 


- Objectives of the auditee 

- Departmental and internal plans 

- Relevant policies and procedures 

- Reporting requirements 

- Services provided 

- Organization chart, position descriptions 
- Capital and operating budgets 


- Performance standards 


Operational Documentation for Audited Entity CG 


- Systems documentation (program logic, management control 


framework, accountability relationships). 


Previous Studies or Reports DD 


. Copies of previous audit reports, follow-up notes, management 


action plans, planning memorandum. 


- Copies of all recent AG reports, departmental responses. 


Internal Audit Handbook 
Volume II, Part 1 
Appendix B ie Mle 


Subject Index 


= Other internal departmental or central agency evaluation group 


studies or reports. 


The foregoing list is meant to be illustrative and not restrictive in any 
way. Additional significant information should be included as 


necessary. 
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VERIFICATION PROGRAMS 


Program A Controls Testing for Essential Controls 
Program B Controls Testing for Control Weakness 
Program C Substantive Testing 


Appendix C 
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AUDIT PROGRAM - CONTRACTING FOR SERVICES 
XYZ DEPARTMENT 


CONTROLS TESTING 
FOR ESSENTIAL CONTROLS 


Verification Objectives: Process Criterion 1 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff activities relating to TB 


guideline 1973-44 are operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff activities relating to TB guideline 


1973-44 are operating effectively. 


Auditing Procedures 


Objective A Done by _W/P ref. 


(initialled (Working 
by auditors) Paper 
reference) 
Select a sample (using statistical sampling techniques) of 
contracts for services processed during a test month of the 


current year. 

For these contracts: 

1) Examine the related contract checklist and determine: 
a) that Part IV of the contract checklist was 


completed and signed off by the Director of 


Personnel Services; 
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Program A 
Objective A (cont'd) Done by __W/P ref. 


b) that Financial Services has ensured that 
Personnel Services reviewed the terms of the 
contract as evidenced by the completion of 


step 9(i) of Part V of the contract checklist. 


2) Determine the nature and extent of information 
provided to the Director of Personnel Services for 
controlling staff activities in relation to TB guideline 
1973-44, 


Assess whether the information provided a sufficient 
basis for determining whether staff performed an 
adequate review of the contract situation in relation 


to TB guideline 1973-44 requirements. 


3) Ensure that any deficiencies noted by the Director of 
Personnel Services were properly resolved on a timely 


basis. 


(Where no deficiencies noted in sample, choose one 


instance from other contracts for testing.) 
Based on the above procedures conclude as to whether 
control over staff activities relating to TB guideline 1973- 


44 is operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by _W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


the Director of Personnel Services as follows: 


a) Review the contract situation and determine 
whether Personnel Services staff obtained all 
necessary information and performed all 


procedures required by TB guideline 1973-44. 


b) Through review of available documentation and 
discussion with staff, critically appraise the 
adequacy of the work performed by the Personnel 
Services staff in relation to the requirements of 
TB guideline 1973-44, 


c) By reference to audit tests on the results 
relating to this process (see Verification Program 
- Results Criterion 1), determine whether any 


improvements in staff activities are required. 


Follow up any deficiencies noted with the Director of 
Personnel Services. (Tests relate to determining effectiveness 


of control in meeting objective set for it.) 


2) Determine the length of time required for the 
Director's review of the work performed by Personnel 
Services and investigate any unusual delays noted. 
(Test to ensure control is timely and involves minimal 


disruption to operations.) 
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Program A 
Objective B (cont'd) Done by _W/P ref. 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Text to ensure 


control procedures are cost-effective.) 
Based on the above procedures conclude as to whether the 
control over staff activities conform to TB guideline 1973- 


44 is operating effectively. 


State any reservations on a separate working paper. 


S 
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Verification Objectives: Results Criterion | 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the control used to detect and correct contracting situations 


that have resulted in labour relations problems is operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the control used to detect and correct contracting situations 


that have resulted in labour relations problems is operating effectively. 


Auditing Procedures 


Objective A Done by _W/P ref. 


1) Identify all labour relations problems which occurred 
during a test month. Determine through discussion 
with the staff responsible for dealing with the dispute 
whether the grievance was fully disclosed to the 


Director of Personnel Services. 
Follow up any deficiencies noted. 

2) Ensure that any items noted for corrective action by 
the Director of Personne! Services were followed up 
on a timely basis. 

Based on the above procedures conclude as to whether the 

control used to detect and correct improper contracting 


situations is operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by __W/P ref. 


1) For the test month selected, assess the effectiveness 
of the control procedures performed by the Director 


of Personnel Services as follows: 


a) Review the nature of all labour relations 
problems that arose. Assess whether any of 
these problems may be attributed to a dispute 
as to whether work contracted-out should have 
been handled by Public Service employees. 
Where such a situation arose ensure that it was 
detected and subjected to appropriate corrective 


action by the Director of Personnel Services. 


b) As part of any necessary corrective action, 
ensure that the Director examined that the 
method by which contracts are processed for 
any deficiencies which may have led to the 
labour problem. (Refer to Verification Program 
- Process Criterion 1.) 

(Test to ensure control meets objective set for 


Te) 


2) Determine the length of time required for the 
Director's review of labour relations problems and 
investigate any unusual delays noted. (Test to ensure 
control is timely and involves minimal disruption to 


operations.) 
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Program A 
Objective B (cont'd) Done by __W/P ref. 


3) | Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective). 


Based on the above procedures conclude as to whether the 
control for detecting and correcting labour relations 
problems relating to improper contracting situations is 


operating effectively. 


State any reservations on a separate working paper. 
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Program A 


Verification Objectives: Process Criterion 2 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff reviews for employer-employee 


relations are operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff reviews for employer-employee 


relations are operating effectively. 


Auditing Procedures 


Objective A i Done by __W/P ref. 


Select a sample (using statistical sampling techniques) of 
contracts for services processed during a test month of the 


current year. 


For these contracts: 


1) Examine the related contract checklist and determine: 


a) that Part IIl of the contract checklist was fully 
completed and signed by the responsibility 


CenEe manager; 


b) that Financial Services has ensured that Legal 
Services reviewed the terms of the contract as 
evidenced by the completion of Step 9(i) of 


Part V of the contract checklist; 


c) that Financial Services has signed off on their 
control review responsibilities as evidenced by 
the appropriate signature on Part V of the 


checklist. 
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Program A 
Objective A (cont'd) Done by __W/P ref. 


2) Determine what additional information was reviewed 
by Financial Services to ensure that Part III of the 
contract checklist was completed by the Manager in 


an appropriate manner. 


Conclude on the adequacy of the information gathered 


for control purposes. 


3) Examine the written legal opinion and ensure that the 
legal review seems reasonable covering all criteria 
noted in TB APM Chapter 312 article .3.3. 


4) Ensure that any contract deficiencies noted by 
Financial Services or Legal Services were properly 


resolved before the contract was let. 


(Where no deficiencies noted in sample, choose one 


instance from other contracts for testing.) 
Based on the above procedures conclude as to whether the 
controls over staff reviews for employer-employee relation- 


ships are operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by _W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


Financial Services and Legal Services as follows: 


a) Review the contract situation and determine 
whether responsibility centre managers obtained 
all necessary information and performed all 
procedures required by TB APM Chapter 312 
article .3.3. 


b) Through review of available documentation and 
discussion with staff, critically appraise the 
adequacy of the review performed by responsi- 
bility centre manages in relation to the require- 
ments of TB APB Chapter 312, article .3.3. 


c) By reference to audit tests on the results 
relating to this process (see Verification 
Program - Results Criterion 2), determine 
whether any improvements in staff activities 


are required. 


Follow up any deficiencies noted with Financial Services. 
(Tests relate to determining the effectiveness of contro! in 


meeting objective set for it.) 
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Program A 
Objective B (cont'd) Done by __W/P ref. 


2) | Through inquiry of responsibility centre managers, 


determine: 


a) their satisfaction with the length of time 
required in finalizing contracts for services 


(test to ensure the timeliness of control). 


b) | whether the review performed by Financial 
Services and Legal Services did not disrupt in 
any significant way the conduct of normal 
operations (test to ensure minimal disruption 


from ccntrol). 


c) | whether they believe that the control review 
can be performed in a simpler or more timely 
manner (test to ensure control procedures are 


cost-effective). 
Based on the above procedures conclude as to whether the 
controls over staff reviews for employer-employee relations 


are operating effectively. 


State any reservations on a separate working paper. 
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Verification Objectives: Process Criterion 3 


A. 


Program A 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether the controls over staff activities required by TB guidelines 


1971-64 and 1971-168 are operating as designed. 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether the controls over staff activities required by TB guidelines 


1971-64 and 1971-168 are operating effectively. 


Auditing Procedures 


Objective A 


Select a sample of the special or unusual contracts to which 
TB guideline 1971-64 and 1971-168 apply. 


For these contracts: 


1) 


2) 


Examine for Financial Services initialling of the 
responsibility centre manager's fee analysis as evidence 


of the control review. 


Determine the nature and extent of information 
provided to Financia! Service for controlling activities 
in relation to TB guidelines 1971-64 and 1971-168. 


Assess whether the information provided a sufficient 
basis for determining the adequacy of the responsibility 
centre manager's analysis in relation to TB guidelines 
1971-64 and 1971-168. 


Done by __W/P ref. 
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Program A 
Objective A (cont'd) Done bya W/Ritet, 


3) | Ensure that any deficiencies noted by Financial 


Services were properly resolved on a timely basis. 


(Where no deficiencies noted in sample choose one 


instance from other contracts for testing.) 
Based on the above procedures conclude as to whether 
control over staff activities relating to TB guidelines 1971- 


64 and 1971-168 is operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by __W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


Financial Services as follows: 


a) Review the contract situation and determine 
whether the responsibility centre manager 
obtained all necessary information and performed 
all necessary tasks in relation to TB guidelines 
1971-64 and 1971-168. 


b) Through review of available documentation and 
discussion with staff, critically appraise the 
adequacy of the work performed by the respon- 
sibility centre manager in relation to TB guide- 
lines 1971-64 and 1971-168. 


c) By reference to audit tests on results relating 
to this process (see Verification Program - 
Results Criterion 3) determine whether any 
improvements in the manager's analyses are 


required. 


Follow up any deficiencies noted with Financial Services. 
(Tests relate to determining the effectiveness of control in 


meeting the objective set for it). 


2) Determine the length of time required for Financial 
Services review and investigate any unusual delays 
noted. (Test to ensure control is timely and involves 


minimal disruption to operations). 
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Program A 
Objective B (cont'd) Done by _W/P ref. 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective). 
Based on the above procedures conclude as to whether the 
control over staff activities conform to TB guidelines 1971- 


64 and 1971-168 is operating effectively. 


State any reservations on a separate working paper. 
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Program A 


Verification Objectives: Results Criterion 3 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over contract fees are reasonable in relation 


to TB guidelines 1971-64 and 1971-168 and operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over contract fees are reasonable in relation 


to TB guidelines 1971-64 and 1971-168 and operating effectively. 


Auditing Procedures 


Objective A Done by __W/P ref. 


Using the contracts selected for testing for Process 


Criterion 3: 


1) Establish evidence which indicates that fees were 
subject to control review by the Financial Services 


Division. 


2) Determine the nature and extent of information 
provided to Financial Services for assessing the 
reasonableness of fees in relation to TB guidelines 
1971-64 and 1971-168. 


Assess whether the information provided a sufficient 


basis for controlling contract fees. 


3) Ensure that any deficiences noted by Financial 


Services were properly resolved on a timely basis. 


(Where no deficiencies noted in sample choose one 


instance from other controls for testing.) 
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Program A 
Objective A (cont'd) Done by _W/P ref. 


Based on the foregoing procedures conclude as to whether 
control over contract fees are reasonable in relation to TB 


guidelines 1971-64 and 1971-168 and operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by __W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


Financial Services as follows: 


a) Review the contract situation and determine 
whether the responsibility centre manager 
considered all necessary factors in arriving ata 


contract fee. 


b) Through review of available documentation and 
discussion with staff, critically appraise the 
reasonableness of the fees arrived at in relation 
to the requirements of TB guidelines 1971-64 
and 1971-168. 


c) As part of any necessary corrective action, 
ensure that Financial Services examined the 
method by which fees were determined as a 


possible cause for any deficiences noted. 
(Refer to Verification Program - Process Criterion 3.) 
(Test to ensure control meets objective set for it.) 
2) Determine the length of time required for Financial 
Services review and investigate any unusual delays 


noted. (Test to ensure control timely and non- 


disruptive.) 


Internal Audit Handbook 
Volume II, Part 1 


Appendix C -lll- 
Program A 
Objective B (cont'd) | Done by _W/P ref. 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective.) 
Based on the above procedures conclude as to whether the 
controls over contract fees are reasonable in relation to TB 


guidelines 1971-64 and 1971-168 and operating effectively. 


State any reservations on a separate working paper. 
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Program A 


Verification Objectives: Process Criterion 4 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff analysis of the appropriate contract 


fee in relation to TB APB Chapter 312 article .4.3.9 are operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over staff analysis of the appropriate contract 


fee are operating effectively. 


Auditing Procedures 


Objective A Done by W/P ref. 


For the contracts previously selected: 


1) — Ensure that the following documentation was prepared 
by the responsibility centre manager and subject to 


control review by Financial Services: 


- the requirements of the task were outlined in 


Appendix D attached to contract checklist 


- qualifications of the individual are documented 


in a curriculum vitae 


- the composition of fees is outlined in Section 7 


of the contract checklist 


- for competitive contracts, all tendering 
information for all bidders was provided to 


Financial Services. 
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Program A 
Objective A (cont'd) Done by _W/P ref. 


2) Ensure that any deficiencies noted by Financial 
Services in relation to the staff analysis of the 
appropriate contract fee was properly resolved on a 


timely basis. 

Where none in sample, choose one example for testing. 
Based on the above audit procedures conclude as to whether 
the controls over staff analysis of the appropriate contract 


fee are operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by __W/P ref. 


1) 


For the contracts selected for testing, assess the 


effectiveness of the control procedures by Financial 


Services as follows: 


a) 


b) 


c) 


Review the contract situation and determine 
whether the responsibility centre manager 
obtained all necessary information and performed 


all required tasks in relation to article .4.3.9, 


Through review of available documentation and 
discussion with staff, critically appraise the 
adequacy of the work performed by the respon- 


sibility centre manager. 


By reference to audit tests on results relating 
to this process (see Verification Program - 
Results Criterion 4) determine whether any 
improvements in the manager's analysis are 


required. 


Follow up any deficiencies noted with Financial Services. 


(Tests relate to determining the effectiveness of control in 


meeting the objective set for it.) 


2) 


3) 


Determine the length of time required for Financial 


Services review and investigate any unusual delays 


noted. (Test to ensure control is timely and involves 


minimal disruption to operations.) 


Assess whether the control review can be performed 


in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective.) 
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Program A 
Objective B (cont'd) Done by __W/P ref. 


Based on the above procedures conclude as to whether 
controls over staff analysis in relation to TB APB Chapter 


312 article .4.3.9 are operating effectively. 


State any reservations on a separate working paper. 
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Program A 


Verification Objectives: Results Criterion 4 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over contract fees are reasonable in relation 


to TB APB Chapter 312 article .4.3.9 and operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over contract fees are reasonable in relation 
to TB APB Chapter 312 article .4.3.9 and operating effectively. 


Auditing Procedures 


Objective A Done by W/P ref. 


Using the contracts selected for testing for Process 


Criterion 4: 


1) Establish evidence which indicates that fees were 
subject to control review by the Financial Services 


Division. 


2) Determine the nature and extent of information 
provided to Financial Services for assessing the 
reasonableness of fees in relation to TB APB 
Chapter 312 article .4.3.9. 


Assess whether the information provided a sufficient 


basis for controlling contract fees. 


3) Ensure that any deficiencies noted by Financial 


Services were properly resolved on a timely basis. 


(Where no deficiencies noted in sample choose one 


instance from other controls for testing.) 
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Program A 
Objective A (cont'd) Done by _W/P ref. 


Based on the above procedures conclude as to whether 
control over contract fees are reasonable in relation to 
TB APB Chapter 312 article .4.3.9 and operating as designed. 


State any reservations on a separate working paper. 
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Objective B 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


Financial Services as follows: 


a) Review the contract situation and determine 
whether the responsibility centre manager 
considered all necessary factors in arriving at a 


contract fee. 


b) Through review of available documentation and 
discussion with staff, critically appraise the 
reasonableness of the fees arrived at in relation 
to the requirements of TB APB Chapter 312 
article .4.3.9. 


c) As part of any necessary corrective action, 
ensure that Financial Services examined the 
method by which fees were determined as a 


possible cause for any deficiences noted. 
(Refer to Verification Program - Process Criterion 4.) 
(Test to ensure control meets objective set for it.) 


2) Determine the length of time required for Financial 
Services review and investigate any unusual delays 
noted. (Test to ensure control timely and non- 


disruptive.) 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective.) 


Done by _W/P ref. 
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Program A 
Objective B (cont'd) Done by _W/P ref. 


Based on the above procedures conclude as to whether the 
controls over contract fees are reasonable in relation to 


TB APB Chapter 312 article .4.3.9 and operating effectively. 


State any reservations on a separate working paper. 
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Program A 
( 
Verification Objectives: Results Criterion 5 
A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over the authorization of the total cost and 
time-rate fees reflected in contracts are operating as designed. 
B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the controls over authorization are operating effectively. 
Auditing Procedures 
Objective A Done by _W/P ref. 
1) Using the sample of contracts previously selected: 
a) | Examine the related contract checklist and 
ensure that Financial Services did a proper ( 


review for authorizations as evidenced by the 
completion of steps 9 and 12 of Part V of the 
contract checklist. 


b) Examine signing authority cards held by 
Financial Services and ensure that they are 


current. 


c) Where any deficiencies were noted by Financial 
Services in relation to authorizations ensure 
that they were properly resolved on a timely 


basis. 
Based on the above procedures conclude as to whether the 
controls over contract authorizations are operating as 


designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by _W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures performed by 


Financial Services as follows: 


a) Examine the authorization made on the contract 
checklist and the contract. Ensure that appro- 
priate authority was received given the total 


cost and time-rate fees involved in the contract. 


b) | Compare authorization made to the signing 


authority cards held by Financial Services. 


Follow up any deficiencies with Financial Services. 
(Tests relate to determining the effectiveness of 


control in meeting objective set for it.) 


2) Determine the length of time required for Financial 
Services review and investigate any unusual delays 
noted. (Test to ensure control is timely and involves 


minimal disruption to operations.) 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective.) 
Based on the above procedures conclude as to whether 
controls over contracts are properly approved are operating 


effectively. 


State any reservations on a separate working paper. 
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Program A 


Verification Objectives: Results Criterion 6 


A. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether the controls over TB submissions are operating as designed. 


B. To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether the controls over TB submissions are operating effectively. 


Auditing Procedures 


Objective A Done by _W/P ref. 


1) For the contracts previously selected: 


a) Examine the related contract checklist and 
ensure that Financial Services did a proper 
review for the completion of a TB submission 
where required. Use Step | of Part V of the 
contract checklist as evidence of the control 


review. 


b) Determine the nature and extent of information 
provided to Financial Services and assess whether 


it was adequate for control purposes. 
c) Where any deficiencies were noted by Financial 
Services in relation to TB submissions ensure 


that they were resolved on a timely basis. 


Based on the above procedures conclude as to whether the 


controls over TB submissions are operating as designed. 


State any reservations on a separate working paper. 
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Program A 
Objective B Done by __W/P ref. 


1) For the contracts selected for testing, assess the 
effectiveness of the control procedures by Financial 


Services as follows: 


a) Review the contract situation and determine 
whether the responsibility centre manager 
obtained all necessary information and performed 


all required tasks in preparing the TB submission. 


b) Through review of the available documentation 
and discussion with staff, critically appraise the 
adequacy of the TB submission in relation to 
TB APB Chapter 312 article .4.3.7. 


Follow up any deficiencies noted with Financial Services. 
(Tests relate to determining the effectiveness of control in 


meeting the objective set for it.) 


2) Determine the length of time required for Financial 
Services review and investigate any unusual delays 
noted (Test to ensure control is timely and involves 


minimal disruption to operations.) 


3) Assess whether the control review can be performed 
in a simpler or more timely manner. (Test to ensure 


control procedures are cost-effective.) 
Based on the above procedures conclude as to whether 
controls over TB submissions in relation to TB APB 


Chapter 312 article .4.3.7 are operating effectively. 


State any reservations on a separate working paper. 
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Program B 


AUDIT PROGRAM - CONTRACTING FOR SERVICES 
XYZ DEPARTMENT 


CONTROLS TESTING 
FOR CONTROL WEAKNESS 


Verification Objective: 


To obtain sufficient, appropriate audit evidence relating to the likely cause 
and effects associated with the noted control deficiency to enable the 


determination of the nature and extent of substantive procedures required. 
Auditing Procedures Done by __W/P ref. 


1) Through inquiry with the auditee identify any reasons 
why control has not been established to detect and correct 
contracting situations that have resulted in employer- 


employee relations. 
Consider, at least, the following possible causes: 


- management did not recognize the need for 
control 
- cost/benefit factors 


- insufficient resources 


2) Determine through inquiry with the auditee, the 


impact of the noted lack of control. 


3) | Select a number of contracts for a test month. 
Through discussion with responsibility centre staff 
and examination of available documentation attempt 
as far as possible to determine whether an employer- 
employee relationship was suggested by the actual 


contract situation. 
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Program B 
Auditing Procedures (cont'd) Done by _W/P ref. 


Based on the above procedures determine the nature and 
extent of substantive procedures required (consider your 
assessment of the adequacy of the related process criterion 


and the possible materiality of errors that may arise). 
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Program B 


Verification Objective 


To obtain sufficient, appropriate audit evidence relating to the likely cause 
and effects associated with the noted control deficiency to enable the 


determination of the nature and extent of substantive procedures required. 
Auditing Procedures Done by __W/P ref. 


1) Through inquiry with the auditee, identify any 
reasons why control has not been established over 
staff analyses of the appropriateness of fees for non- 


competitive contracts. 
Consider, at least, the following possible causes: 


- management did not recognize the need for 
control 
- cost/benefit factors 


- insufficient resources 


2) Determine through inquiry with the auditee, the 


impact of the noted lack of control. 


Consider, at least, the likelihood and magnitude of 


contract fees exceeding the going market rate. 


3) Select a sample of non-competitive contracts for a 
test month. Assess the reasonableness of contract 
fees. This may be performed by comparing the fees 
with the "going rate" for a particular expertise. 
Information on such rates may be available from the 
department or agency or from other clients in the 


public and private sectors. 
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Program B 
Auditing Procedures (cont'd) Done by _W/P ref. 


Based on the above procedures determine the nature and 
extent of substantive procedures required (consider your 
assessment of the adequacy of the Manager's review and 


the possible materiality of errors that may arise). 
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Program C 


AUDIT PROGRAM - CONTRACTING FOR SERVICES 
XYZ DEPARTMENT 


SUBSTANTIVE TESTING 
Verification Objective: Process Criterion 1 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether staff activities comply to TB guideline 1973-44, 


Auditing Procedures Done by _W/P ref. 


Where controls are effective and the auditor wishes to rely 


on them: 


Select a limited sample of contracts from the entire period 


under review and perform the following tests: 


1) Review the contracts generally and ensure that the 
relevant control procedures were applied; having 


established that the control was in place: 


a) Review the contract situation and determine 
whether Personnel Services obtained all 
necessary information and performed all tasks 
required by TB guidelines 1973-44. 


b) Through a review of available documentation 
and discussion with staff critically appraise the 
adequacy of the work performed by Personnel 


Services staff in relation to TB 1973-44, 


Follow up any unusual items. 
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Auditing Procedures (cont'd) 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of contracts from the 
entire period under review and perform steps a) and 


b) on previous page. 
Based on the above procedures conclude whether staff 
activities comply to the requirements of TB guideline 


1973-44, 


State any reservations on a separate working paper. 


Program C 


Done by __W/P ref. 
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Program C 


Verification Objective: Results Criterion 1 
To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether contracting situations do not result in labour relations 


problems. 
Auditing Procedures Done by __W/P ref. 


Where controls are effective and the auditor wishes to rely 
on them: 


Select a limited sample of labour relation grievances 
reported throughout the entire period under review and 


perform the following test: 

1) Review the nature of the labour relations problem 
and assess whether it can be attributed to a dispute 
as to whether work contracted-out should have been 


handled by Public Service employees. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of labour relations problems 


and perform Step | above. 


Based on the above procedures conclude whether contracting 


situations do not result in labour relations problems. 


State any reservations on a separate working paper. 
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Program C 


Verification Objective: Process Criterion 2 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether staff reviews for employer-employee relations are 


adequately performed before a contract for service is let. 
Auditing Procedures Done by _W/P ref. 


Where controls are effective and the auditor wishes to rely 
on them: 


Use the limited sample of contracts selected for Process 


Criterion | and perform the following tests: 


1) Through review of available documentation and 
discussion with staff, critically appraise the adequacy 
of the review performed by responsibility centre 
managers in relation to the requirements of TB APB 
Chapter 312 article .3.3. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of contracts from the 


entire period under review and perform step | above. 
Based on the above procedures conclude whether staff 
reviews for employer-employee relations are adequately 


performed before a contract for service is let. 


State any reservations on a separate working paper. 
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Verification Objective: Results Criterion 2 


Program C 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether employer-employee relations are not established in 


contracts for services. 


Auditing Procedures 


. Where controls are effective and the auditor wishes to rely 
on them: 


Use the limited sample of contracts selected for Process 


Criterion | and perform the following tests: 


1) Through inquiry with auditee staff, review of 
contractor outputs and examination of any other 
evidence relating to the specific conduct and outputs 
of the contract under review, assess whether this 
information suggests the existence of employer- 


employee relations. 


Follow up any unusual items. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of contracts from the 


entire period under review and perform step | above. 
Based on the above procedures conclude whether employer- 
employee relations have not been established for contracts 


under review. 


State any reservations on a separate working paper. 


Done by _W/P ref. 
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Program C 


Verification Objective: Process Criterion 3 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether staff consider all factors involved in establishing a fee for 
contracts covered by TB guidelines 1971-64 and 1971-168. 


Auditing Procedures Done by _W/P ref. 


Where controls are effective and the auditor wishes to rely 
on them: 


Select a limited sample of special or unusual contracts 
covered by TB 1971-64 and 1971-168 from the entire period 
under review and perform the following tests: 


1) Examine the documentation prepared in the 
establishment of a contract fee. Assess the adequacy 
of staff analysis of the contract fee in line with 
requirements of TB guidelines 1971-64 and 1971-168. 


Follow up any deficiencies noted. 


Where controls are missing or defined and the auditor 
cannot rely on them: 


2) Select an extended sample of special or unusual 


contracts and perform step | above. 
Based on the above procedures conclude as to whether 
staff activities conform to the requirements of TB 


guideline 1971-64 and 1971-168. 


State any reservations on a separate working paper. 
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Program C 


Verification Objective: Results Criterion 3 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether contract fees are reasonable in relation to TB guidelines 
1971-64 and 1971-168. 


Auditing Procedures Done by __ W/P ref. 


Where controls are effective and the auditor wishes to rely 


on them: 


Use the limited sample selected for Process Criterion 3. 


Perform the following test: 


1) Critically appraise the reasonableness of contract 
fees. Use the criteria set out in TB guidelines 1971- 
64 and 1971-168 as a guide. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of special or unusual 
contracts from the entire period under review and 


perform step | above. 
Based on the above procedures conclude whether contract 
fees are reasonable in relation to the requirements of TB 


guidelines 1971-64 and 1971-168. 


State any reservations on a separate working papers. 
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Verification Objective: Process Criterion 4 


Program C 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether staff consider all relevant factors in establishing the 


appropriate fee for each contract for services. 


Auditing Procedures 


Where controls are effective and the auditor wishes to rely 
on them: 


Use the limited sample of contracts selected for Process 


Criterion 1. Perform the following test: 


1) | Examine the documentation prepared in the establish- 
ment of an appropriate contract fee. Assess the 
adequacy of the staff analysis of the contract fee in 
line with the requirements set out in TB APM 
Chapter 312 article .4.3.9. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


Select an extended sample of contracts from the entire 


period under review and perform the following test: 


2) | Examine the documentation prepared or inquire as to 
other analysis performed in the establishment of an 
appropriate contract fee. Assess the adequacy of the 
staff analysis of the contract fee in line with the 
requirements set out in TB APM Chapter 312 
article .4.3.9. 


Done by W/P ref. 
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Program C 
Auditing Procedures (con't) Done by _W/P ref. 


Follow up any deficiencies noted. 
Based on the above procedures conclude as to whether 
staff consider all relevant factors in establishing the 


appropriate fee for each contract for services. 


State any reservations on a separate working paper. 
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Verification Objective: Results Criterion 4 


Program C 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether contract fees are reasonable in relation to TB APB 


Chapter 312 article .4.3.9. 


Auditing Procedures 


Where controls are effective and the auditor wishes to rely 
on them: 


Use the limited sample of contracts selected for Process 


Criterion 1. Perform the following test: 
1) Critically appraise the reasonableness of the contract 
fees. Use the criteria set out in TB APB Chapter 312 


article .4.3.9 as a guide. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


Select an extended sample of contracts from the entire 


period under review and perform step | above. 


Based on the above procedures conclude whether contract 


fees are reasonable in relation to the requirements of TB APB 


Chapter 312 article .4.3.9. 


State any reservations on a separate working paper. 


Done by _W/P ref. 
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Verification Objective: Results Criterion 5 


Program C 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether the total cost and time-rate fees reflected in contracts 


for services are properly authorized. 


Auditing Procedures 


Where controls are effective and the auditor wishes to rely 
on them: 


Use the limited sample of contracts selected for Process 


Criterion |. Perform the following test: 


1) Examine the authorizations made on both the contract 
checklist and the contract. Ensure that appropriate 
authority was received, given the total cost and time- 
rate fees involved in the contract. Compare authoriza- 
tions made to the signing authority cards held by the 
department. 


Follow up any deficiencies noted. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


2) Select an extended sample of contracts from the 


entire period under review and perform step | above. 
Based on the above procedures conclude as to whether the 
total cost and time-rate fees reflected in contracts for 


services are properly authorized. 


State any reservations on a separate working paper. 


Done by _W/P ref. 
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Program C 


Verification Objective: Results Criterion 6 
To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether Treasury Board submissions include all necessary 
information. 


Auditing Procedures Done by __W/P ref. 


Where controls are effective and the auditor wishes to rely 
on them: 


Select a limited sample of contracts requiring a TB 
submission from the entire period under review and perform 


the following tests: 

1) ensure that Financial Services performed a control 
review as evidenced by the completion and signing 
off of Part V of the contract checklist; 

2) having established the control was operating: 

- examine related TB subrnission and ensure that 
they include all information as outlined in TB 


APM Chapter 312 article .4.3.7. 


Follow up unusual items. 
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Program C 
Auditing Procedures (cont'd) Done by _W/P ref. 


Where controls are missing or deficient and the auditor 
cannot rely on them: 


3) Select an extended sample of contracts requiring TB 
submission from the entire period under review and 


perform step 2. 


Based on the above procedures conclude as to whether TB 


submissions include all necessary information. 


State any reservations on a separate working paper. 
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Program C 


Verification Objective: Process Criterion 7 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 
formed on whether the auditee has undertaken an adequate study for ensuring 


that submissions to TB are made as efficiently as possible. 


Auditing Procedures Done by __W/P ref. 


1) Through discussion with the auditee determine the 
nature of any study performed for expediting the 
submissions to TB. Using the points made in 
TB APB Chapter 312 article .4.3.8 assess the adequacy 
of the study undertaken by the auditee. 


Follow up any deficiencies noted. 
Based on the above procedures conclude whether the auditee 
has undertaken an adequate study for ensuring that TB 


submissions are handled efficiently. 


State any reservations on a separate working paper. 
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Program C 


Verification Objective: Results Criterion 7 


To obtain sufficient, appropriate audit evidence to enable a conclusion to be 


formed on whether TB submissions have been made efficiently. 


Auditing Procedures Done by _W/P ref. 


1) Selection an extended sample of TB submissions from 
the entire period under review. Assess whether 
submissions made could have been aggregated into an 


omnibus submission to improve efficiency. 
Follow up any deficiencies noted. 


Based on the above procedures conclude whether TB 


submissions are made in an efficient manner. 


State any reservations on a separate working paper. 


LOWE-MARTIN 


LOWE-MARTIN 86-125 
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PART TWO: INTERNAL AUDIT CONCEPTS AND APPROACHES 
CHAPTER ONE 
INTERNAL AUDIT APPROACHES: OBJECTIVES AND LINES OF INQUIRY 
INTRODUCTION 


This chapter supplements Part 1 which deals with the internal audit assignment 
process. Specifically, this chapter elaborates the relationship between the purpose 
and scope of an audit assignment and the corresponding, detailed audit objectives 


and associated lines of inquiry. 


As with audit purpose and scope, audit objectives and therefore lines of inquiry 
depend on the type of audit being undertaken and the subject matter of the specific 
audit unit being audited. Accordingly, Section One describes the relationship 
between purpose, scope, audit objectives and lines of inquiry for a number of typical 


types of audit that may be undertaken. 


In undertaking an audit assignment there are general principles which apply in 
choosing an audit strategy. Some of these are enumerated and discussed in 
Section Two. Also, the trade-off between maximizing advice to management and 
efficiency in planning for and executing an audit assignment, or a series of 


assignments, is dealt with. 
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SECTION ONE: AUDIT OBJECTIVES AND LINES OF INQUIRY FOR 
TYPICAL TYPES OF AUDITS UNDERTAKEN 


In this chapter, the purpose and scope for each type of audit will be carried forward 
from Part 1, Volume II, of this Handbook for the purpose of continuity. The object 
of this section is the establishment of a clear connection or bridge between purpose 


and scope on the one hand, and objectives and lines of inquiry on the other. 


Each type of audit will be described in turn, including its particular purpose and 
scope, and the relationship between the general and specific purpose, scope and 


objectives for internal auditing. 


The Standards!, supplemented by Part 1, Volume II of this Handbook, provide 
generic purpose (role), scope and objectives for internal auditing (see 
Tables | and 2). 


As is the case for purpose and scope, the specific objectives set and lines of inquiry 
chosen for any particular internal audit assignment depend on the type of audit 
undertaken (i.e. Responsibility Centre (RC) Audit, Organization Audit, Function 
Audit, Systems Audit, Special Audit). However, in the case of objectives, and even 
more so for lines of inquiry, two other factors determine their extent and nature. 
These are the subject matter of the audit (e.g. program or activity, function, 
system) and its context (e.g. centralized or decentralized, volatile or stable, 


material or high risk, or non-material or low risk). 


The audit purpose and scope statements indicate why an internal audit is undertaken 
and where. The audit objectives state what performance the auditor wishes to 
assess. Finally, lines of inquiry delineate how the assessment is going to be carried 


out. 


1 Standards for Internal Audit in the Government of Canada, Treasury Board of 
Canada, Office of the Comptroller General, 1982. 
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(A) 


(B) 


Table | 


Purpose and Scope of Internal Auditing 


Purpose (Role) of Internal Auditing 

"Departments shall have an independent internal audit function that carries 
out a systematic review and appraisal of all departmental operations for 
purposes of advising management as to the efficiency, economy and effective- 
ness of internal management policies, practices and controls." 


Scope of Internal Auditing 


"The scope of internal audit shall encompass all aspects of a department's 


operations. The internal auditor assesses and expresses an opinion upon: 


® the design, development, implementation and operation of all systems, 


procedures, processes and controls, including computer-based systems; 


v7 the reliability and adequacy of information available for decision-making 


and for accountability purposes; 


e the extent to which available information is utilized in the decision- 


making process; 
® the adequacy of protection afforded public funds and assets; and 


® the extent of compliance with legislative, central agency and 


departmental direction." 


Ibid, p. 4. 
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(C) 


Table 2 


General Internal Audit Objectives 


Audit Objectives 


"The objectives of internal auditing include assessing: 


® the integrity of financial and other information; 
e the adequacy of controls over public property, revenues and expenditures; 
a the degree of compliance with objectives, policies, plans, procedures, 


laws and regulations; and 


@ the extent to which there is management with due regard for economy, 


efficiency and effectiveness." 


Ibid, p. 64. 
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In the broadest terms, the objectives of the internal auditor are to assess the 
performance of the audit unit to determine whether required results are being 
achieved, i.e. that the unit is economic, efficient and effective; and that those 
results will continue to be achieved in the future, i.e. that the delivery system is 
performing as intended and is likely to continue to do so. This assessment is 


generally made through the auditee's control framework. 


Lines of inquiry are chosen so that the purpose of the audit is fulfilled over the 
range of activities desired while meeting the specific objectives of the audit and 


doing this efficiently. 


In deciding on lines of inquiry, the following general conditions should be kept in 


minds: 


® With the exception of special audits and those cases where controls are 
known to be non-existent or weak, it is usually more efficient to audit 
the performance of controls rather than audit the operations and results 
directly. Moreover, where controls are sound, it is efficient to rely on 
them, thus minimizing the time-consuming and costly substantive testing 


that would otherwise have to be performed. 


a Although management and staff are distinct, in terms of role, it is 
generally more efficient to audit organizational arrangements (e.g. 
organization structure, job descriptions, delegation documents) as one 
structure rather than as two. That is, it is more efficient to treat them 


as a Single line of inquiry. 


e Although results objectives (i.e. economy, efficiency and effectiveness) 
are generally associated with management and operations outputs with 
the delivery process (i.e. input, process, output), the controls for them 


may overlap. 


® Where policies, directives, procedures, etc. do not exist for an activity 
(e.g. where the activity is rarely performed, straightforward, non- 


standard or it is not cost-effective to develop direction for it) the 
delivery system process controls will be those for management's 


organizing/leadership process. These are generally the least documented. 
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ILLUSTRATION OF MANAGEABLE AUDIT UNITS 
IN TERMS OF TYPES OF AUDITS 


Auditable Units 
1 Z 3 4 5 6 


Responsibility Centres (RCs) 


Department/Agency 
ir Branch (Program A) Is a e valle am 
1.1 Division (Activity A) Vic! | | | 
tial Directorate | | | | | | 
1.1.2 Directorate | | | s l | s | . | 
1.1.3 Directorate | A al ih a 
ESI | Iw! 
M M M 
Tale: Mel tel ie | 
itz Division (Activity B) | L | IL | L | 
d2en Directorate | | l | 
12.2 Directorate E | E E | 
1.2.3 Directorate [ | | | | | 
| Directorate l | | | | | 
BAG | 
Branch (Program B) | | | | l | | 
2.1 Division (Activity C) i | | [ma] 
2.2 Division (Activity D) ioe [ rewaniny 21 | 
[ae 
3, Branch (Administration) a | | 
at Division (Finance) | | | 
= Directorate (Accounting) | | | 
3.2 Division (Personne!) | | 
Seal Directorate (Staffing) | | | l 
3.3 Division (Systems Development) | | _ 
ae 
Legend 
Types of Auditable Units Description 
1 MAU RC Audit 
2 RU/MAU RC/Organization/Program Audit 
3  RU/MAU RC/Organization 
4 RU/MAU Organization/Function Audit 
5 RU/MAU Function Audit 
6 RU/MAU Systems Audit 


MAU = Manageable Audit Unit 
RU = Reportable Unit 


Figure | 
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Responsibility Centre Audit 


A responsibility centre (RC) audit consists of an audit of all the important 
activities undertaken by that RC (whether the RC be program, administrative or a 
combination of the two, e.g. a region), the management of that RC, and the impact 
of functional direction (e.g. program policies, personnel policy, financial policy) on 
the activities of that RC. 


Columns | and 2 of Figure | represent responsibility centres which are considered 
significant. They are, however, not necessarily both reportable and manageable 
audit units. That is, they are manageable as individual audit assignments but may, 
or may not, be worth reporting to senior management on their own merits (i.e. 
Reportable Units). 


In this situation the major auditee is senior management. Derivative auditees would 
be immediate and intermediate management and functional groups which impinge 
on the responsibility centre. Column | of Figure 1 represents a responsibility centre 
audit which may not be, by itself, sufficiently significant to report to senior 
management. In this case, it would probably be rolled up, or grouped, with other 
similar manageable audit units for reporting to senior management as part of a 


reporting audit unit, but would be reported on, as is, to local management. 


The purpose statement for RC audits is derived from the general purpose statement 
in the Standards. It delineates which type of operation will be examined and assessed. 


It may be stated as follows: 


Purpose: This responsibility centre (RC) audit is undertaken for the purpose of 
providing management with advice on the performance of all the important activities 
undertaken by this RC, its management, and the impact of functional direction on 


its activities. 
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When a specific RC is identified as the subject of the audit, its title and/or number 


may be substituted wherever "RC" occurs. 


The scope statement is derived from the general scope statement (see the Standards)’; 
the details are only filled in when the specific RC to be audited is identified. Its 


opening paragraph might read as follows: 


Scope: The scope of this RC audit encompasses all aspects of the RC which fall 
within its organizational boundaries. It is recognized, however, that substantive 
testing activity may require the auditor to obtain data from outside the RC (i.e. 


the environment). Specifically, the scope of this audit includes: .... 


It is not expected that every element of the generic scope statement will be 


applicable to every specific audit. 


Audit Objectives: A responsibility centre (RC) is an auditable unit that contains 
two distinct elements as objects of auditors' attention. One element is the delivery 
system which includes organization structure, policies and procedures, processing 
and associated information systems (both line and functional) along with their 
respective operating controls. The second element is the management system 
which includes the management structure, management process and associated 


management controls. In generic terms these elements include: 
ls Delivery System 
1.1 To assess the performance of the delivery system's static elements: 


(i) organization structure and associated elements (e.g. job 


descriptions, delegation documents), 


(ii) operating systems, policies and procedures, etc. 


4 Ibid, p. 4. 
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1.2 To assess the performance of the delivery system's dynamic elements: 
(i) processing of goods or services, 
(ii) operating output or results. 
2. Management System 
2.1 To assess the performance of the management system's static elements: 


(i) management structure and associated elements (e.g. job 


descriptions, delegation documents), 
(ii) management systems, plans, policies and procedures, etc. 
2.2 To assess the performance of the management system's dynamic elements: 
(i) planning, organizing, leading, controlling, communicating, 


(ii) organizational (RC) outputs or results: economy, efficiency 


and effectiveness. 
Lines of Inquiry: As in most audits, the general approach will be to assess the 
control framework and rely on its feedback, to the degree possible, in deciding on 
the extent of substantive testing required. Accordingly, the lines of inquiry ina 
typical RC audit will include these elements. 
1. Delivery System/Management System 


1.1 Assess the organizational control framework, including: 


- organization structure 


span of control (e.g. adequacy of supervision) 


- reporting relationship 


job descriptions 
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2. 


- delegation documents 
- assignment of responsibilities (e.g. separation of duties) 


- performance review. 


1.2 Assess delivery process controls 
(based on major or significant RC activities). 
1.3. Assess relevant administration process controls: 
- financial 
- personnel 
- administrative (EDP, materiel, property, etc.) 
- other. 
1.4 Assess management process controls: 
(use Guide to an Audit of the Management Process) 
- planning 
- organizing (those not covered in 1.1) 
(a) development of required delivery system 
(b) operation of the delivery system 
~ leading 
- controlling 
- communicating. 
Results 
2.1 Assess delivery system output controls (specific, depend on the particular 
RC's activities). 
2.2 Assess management results controls: 


(specific, depend on the particular RC) 


- economy 
- efficiency 


- effectiveness. 
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Organization Audit 


Organization audits are RC audits where more than one, or parts of more than one, 

RC are involved (e.g. a division, branch, a region, an airport, a ship) (see Column 3 

of Figure 1). In these cases, significance would normally be assigned to the highest- 
level RC which would therefore become the reporting unit as far as senior management 
is concerned, The key distinction between a responsibility centre audit and an 
organization audit is that an organization audit will typically include, for example, 

a number of regional offices within a branch or a number of local offices within a 


region. 


Intermediate- and primary-level line managers would receive summary reports 
(management letters) or complete reports, depending on which of them is considered 
to be the appropriate level to take desired corrective actions. A similar strategy 
would be applicable to peer-level managers who provide functional direction, advice, 


or service to management in the audit unit. 


Purpose: This organization audit is undertaken for the purpose of providing 
management with advice on the performance of a group of responsibility centres, 
or parts of responsibility centres and on the impact of functional direction on their 


activities. 


Scope: The scope is determined by the combined boundaries of the RCs, or parts of 
RCs, included in the audit unit. It is recognized, however, that substantive testing 


activity may require the auditors to obtain data from outside the audit unit. 


Audit Objectives: Since the organization audit is essentially a responsibility centre 
type of audit involving more than one, or parts of more than one, responsibility 


centre, its objectives are identical to those for RC audit except for complexity. 


The complexity of an organization audit derives from the usually hierarchical nature 
of organizations. When auditing a RC which has several subordinate RCs with 
complex operations (i.e. several interrelated activities), complexity for the auditor 
arises not only from the complex operations but from the fact that to any level of 


RC higher than the first level, the operations include both the lower-level RCs and 
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their management. That is, the auditor has to consider both the integrated process 
and the integrated results of the lower-level RCs in developing audit objectives. 
(See Figure 2 for a pictorial representation of this concept, and the associated 


discussions in Section Three of Chapter 2 and Section Two of Chapter anc 


Lines of Inquiry: As in the case of audit objectives, lines of inquiry for organization 
audits are similar to that of an RC audit with one variation. In the case of many 
homogeneous RCs working under a higher-level RC (e.g. several regions reporting 
to a general manager, ADM, or VP of regional operations, or several local offices 
reporting to a regional manager) a statistical approach to audit evidence gathering 
may be adopted. In these cases, data from a few of the regional or local RCs are 


gathered and used to represent all RC's of the same type. 
Function Audits 


A function RC (e.g. finance, personnel, or a group providing functional direction to 
a program) is one which provides functional direction, advice, and sometimes 
services to other RCs within the organization. Generally, direction is provided 
through the issue of formal policies and directives and other similar mechanisms. 
The function audit must include review of the effectiveness, efficiency and economy 
of the direction or service being provided by the functional group and the extent of 
adherence to its policies. To avoid duplication, the audit may be accomplished by 
reviewing pertinent information from recently completed RC or organization audits. 
Column 5 of Figure | illustrates a function audit of the personnel function, while 
Column 4 illustrates a combined organization-function audit, since it includes an 


audit of branch management as well as of the financial function. 


5 Internal Audit Handbook, Volume II, Part 2, Chapter 2, "Control: Concepts 
and Applications for Internal Auditors". 


6 Internal Audit Handbook, Volume II, Part 2, Chapter 3, "Management Control: 
Concepts and Practices". 
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MODELLING, USING THE TECHNIQUE OF PROGRESSIVE ELABORATION 


MODELA 


OUTPUTS 


MODEL B 


i 


OUTPUTS 


Figure 2 
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For a function audit, the major client, in addition to the Deputy Head, is the head 
of the function subject to audit. He or she is also the main auditee. Derivative 
auditees are other, lower-level groups within that function and the selected RC 


managers that were part of the sample. 


Purpose: This audit provides advice to management on an organizational unit which 
provides functional direction advice and/or services to other organizational units 
and includes both program and support (finance, administration, personnel, official 
languages, etc.) functional activities. This functional direction will usually be 
provided through the provision of formal policies, directives, systems and procedures 


and other similar direction and advice or services. 
Scope: Function audits are usually performed on two levels: 


© Audit of the organizational unit which is the source of functional 
direction (the scope of this part of the audit is similar to that of an 


organization or RC audit); and 


a Audit of adherence to policy and directives and use of advice, systems 
and procedures by the organizational units which are subjected to 
functional direction, and of the effect of that direction, advice or 
service on those units (the scope here is all the organization's units that 


are subject to the direction, advice or service provided). 


Audit Objectives: 


® Audit of the RC or organization providing the functional direction, 
advice or service. This part of the audit will have objectives similar to 
that of an organization or RC audit with special emphasis on controls 
which enable managers to determine the degree to which their direction, 


advice or service is having the desired effect; and 


e Audit of RCs or organizations which are subject to the functional 
direction, advice or service in question (i.e. recipient RCs or 


organizations), The audit objective here is to assess the performance of 
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functional directives, advice or services in the context of the needs of 
the recipient RC or organization. Direction, advice or service provided 
will not be of equal relevance or usefulness to all RCs or organizations; 
lines of inquiry and audit criteria will have to be adjusted accordingly 


for each audit assignment. 


Lines of Inquiry: 


Audit of the originating RC or organization: similar to any RC or 
organization audit, whichever is more representative of the functional 


group's organization structure, 


Audit of recipient RCs or organizations: in deciding on the audit approach 
for this part of the audit, decisions will have to be made as to which 
elements of functional direction, advice or service are relevant and 

their relative significance to the various RCs in the auditee organization. 
Those considered both relevant and significant should be included in 


recipient RC audits. 


Relevant evidence may be obtained from RCs in a number of ways: 


(i) 


(ii) 


(iii) 


through 100 per cent audit of all recipient RCs; 


through an audit of a sample of recipient RCs; or 


by accumulating relevant evidence from RC or organization audits 
performed for other purposes, provided they were performed within a 
reasonable time frame prior to the function audit in question, such that 
the evidence is not considered stale (i.e. not representative of the 


current state of the recipient RC). 


It is, of course, necessary to include appropriate scope, objectives and lines of 


inquiry adjustments in the audit assignment plans of those RC or organization audits 


that are intended to provide dual purpose results as proposed in (ii) and (iii). 
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Program Audits 


A program audit is a responsibility centre audit of the RC or RCs charged with the 
overall responsibility for the program or activity (e.g. a branch) and, where 
applicable, an audit of all or a sample of the program-related operations of the 
responsibility centres (e.g. regions, local offices) involved in the delivery of the 


program. 


Depending on how program delivery is organized, a program audit may resemble an 
organization audit, a responsibility centre audit or a function audit. Column 2 of 
Figure | illustrates a program audit. It should be noted that RC 2 (in Figure 1, 
Column 2), which has overall responsibility for the program, is audited as a respon- 
sibility centre audit. Although it would be possible to audit only specific program 
activities in this RC, the administrative and management practices within this RC 
would be expected to affect overall program performance. It is therefore considered 


desirable to audit this type of RC in its entirety as part of a program audit. 


A complication occurs when auditing a regional or local office which delivers more 
than one program from or through a common administrative framework. Here a 
similar audit strategy to that outlined in the foregoing discussion may be employed 
(for example, an audit of a large RC with several activities and its own adminis- 


trative services). 


Purpose: This audit provides advice to management on the performance of a program 


or sub-program (activity) in terms of its delivery system and results. 

Scope: The scope of this audit includes, at the minimum, the delivery system of 
the program or activity and its management. Other elements that may be included, 
depending on how program delivery is organized are: 


e the RC or organization providing functional (policy) direction; and 


& regional or local offices which participate in the delivery of the program 


in whole, or in part (program under audit only). 
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Audit Objectives: Audit objectives for a program audit may resemble those of an 
RC audit, organization audit or function audit depending on how program delivery 
is organized (see the three previous audit objectives sections). Particular attention 
should be given to the existence of program effectiveness controls (i.e. managerial 
controls and program evaluation). 


Lines of Inquiry: As with audit objectives, the lines of inquiry depend on how 


program delivery is organized. They will also depend on audit strategy. 


Audit strategy decisions can occur at several levels. At the head office level, 

where more than one program or activity is being managed there is a decision to be 
made whether only the program or activity under audit will be included in the assign- 
ment or whether the opportunity will be used to provide feedback to management 

on the whole RC. This situation is more likely to occur where a sub-program or 


activity is being audited. 


Similar decisions need to be made where program delivery is through a number of 


regional or local offices. The options to consider include: 


(i) audit 100 per cent of the RCs that act as the delivery vehicle for the 
program under audit, even though they may be delivering other programs 


as well; 
(ii) audit a sample of RCs that deliver the program under audit: 


(a) 100 per cent 


(b) desired program only; 


(iii) audit only that portion of the RCs that deliver the program under audit; 


or 


(iv) rely on audits of RCs which were performed as RC audits or as program 
audits of other programs, which were completed within a sufficiently 


short time frame that the data are still current. 
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Systems Audit 


Although the term "systems audit", if one were to take the most generic definition 
of the word "system", would include all types of audit units already described, the 
term as used here includes only systems in the narrower sense as used by EDP or 


systems and procedures staff. 


A systems audit is an audit of a system used by any functional or line (i.e. program) 


unit, or any combination of the two. 


Column 6, Figure | provides an illustrative example of this type of auditable unit, 


where the Systems Development Division is the host unit. 


Purpose: To provide management with advice on the performance of a user infor- 


mation system, usually EDP-based. 


Scope: This type of audit is similar in nature to the functional audit in that it is 


carried out by performing: 


(i) a review of the organizational unit which is responsible for the overall 
design, development and maintenance of the system (the administering 


unit, typically the EDP group); 


(ii) a review of the organizational unit which hosts (owns), as well as uses, 


the system; and 


(iii) a review of the users of the system, if they are not the owners. 
Note: It is possible for the first two, or all three, to be the same organizational 
unit. 


Audit Objectives: In a systems audit the auditor, typically, is not auditing complete 
responsibility centres, but rather parts of them. However, the general controls 


applicable to all activities in an RC should be reviewed. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 1, Section 1 - 161 - 


The general objectives for a systems audit would include those that the auditor 

would use in a functional audit, remembering that the relevant static and dynamic 
elements of the delivery system may be spread over more than one organizational 
unit (e.g. the EDP and the financial services groups, and RC managers or financial 


officers, in the case of a financial reporting system). 
Specifically, audit objectives could include: 
1. Delivery System 
1.1 To assess the performance of the delivery system's static elements: 

(i) organization of the developer, custodian, host, or users of 
the system, including organization structure, job descriptions, 
delegation documents, quality of personnel, separation of 
duties, security, training, etc.; 

(ii) operating and maintenance systems, policies and procedures 
(including manual procedures, computer programs, users 
manuals, maintenance manuals, etc.); 

(iii) environmental or general controls (including security, data 
integrity, operating system hardware, software or storage 
controls, development or maintenance process controls, 
etc.). 

1.2. To assess the performance of the delivery system's dynamic elements: 


(i) processing of transactions; 


(ii) operating outputs or results. 
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2 Management System 
Note: Unless the system is large enough to warrant a full-time manager 


(i.e. being treated as an RC) this section would not be required, 
but where it is large enough to be treated as an RC, the RC audit 
objectives apply. 


Lines of Inquiry: As indicated in the audit objectives section, the auditor's attention 
should be directed to three areas of activity; namely, the developer, maintainer or 
custodian of the system, the host or owner of the system and the users of the system. 
These areas may not be organizationally distinct, depending on the nature of the 
information system under audit. Accordingly, specific lines of inquiry may include 


the following: 


Ls Audit of the developer, maintainer or custodian of the system: several 


organizational arrangements for this activity are possible: 


1.1 Systems Development Function/RC 


If systems development is expected to be an on-going activity the 
department will likely have established a permanent systems and 
procedures function or RC. If most of the systems development 
activity is EDP-based, and is sufficiently extensive, then it will likely 
be an EDP systems development group situated in an organizationally 
distinct informatics group or computation centre, otherwise it may be 
in the administration branch. The two possibilities are, of course, not 


mutually exclusive. 


Given a distinct group for systems development, the audit will consist 
of an RC or organization audit with the delivery system being a systems 
development structure and process for which there is central agency 


guidance, likely supplemented by departmental guidance as well. 
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1.2 System development is one activity of the host organization. In this 
case an organization or RC audit is carried out of the host organization 


where at least three delivery system elements will be of interest: 
(i) the development activity and its supporting structure; 


(ii) the owner activity that provides direction for, and depends 


on, the information system under audit; and 


(iii) the general, relevant organization structure and processes, 
common to some or all activities, that support both of the 


foregoing activities. 
1.3. Systems development is a minor activity of the host or user group. 


In this case, whether the systems development delivery system is audited 


as a distinct entity or not will depend on its materiality or significance. 


In cases where systems development is audited as one of several activities in 

an organization or RC it will usually be more efficient to audit the complete 

host RC and carry forward findings which are not related to systems development 
for reporting separately, or to incorporate them in reports on adjacent audited 


entities. 
Audit of the owner, host organization or RC 


The owner or host organization is generally the organization unit that 
developed the system, including the user specifications, and continues to 
provide guidance on its use: use, that is, of output as opposed to technical 
operating instructions, which are usually provided by the developers. It is 
also usually a major, if not the major, user. It will generally be the most 
important auditee in determining the economy, efficiency and effectiveness 


of an information system. 
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In this case, audit of the management system will usually take the form of an 
organization or RC audit. Audit of the delivery system will depend on whether 
the information system is the main delivery system of that organization or 
RC, an input to the main delivery system, or a support information system 

for administrative decision-making (e.g. in support of financial, personnel 


decisions). 


In the case of a main delivery system, the audit becomes a program audit, 
unless the host or owner is a functional group. In the latter case it becomes 


part of a function audit. 


If the information system is not a main delivery system, but an important 
input to one, the strategy may be to perform the audit in conjunction with 
the delivery system it feeds. Whether the audit is done individually or 
separately will depend on the relative materiality or significance of the 
respective systems involved and on the degree of interdependence. A support 
information system will usually be best audited as an element of the 


organization or RC that it supports. 
3s Audit of the User of the System 


Situations where the user is also the developer or owner of the system have 
been already dealt with. Where the users are neither developer nor owner, 
they are typically many and dispersed. Audit of their use of the system will 
generally be very similar in nature to an audit of the recipients of guidance in 


a functional audit, i.e. the audit will either be: 


3.1 an already scheduled organization or RC audit of which an element is 
use of the system under audit. In this case the findings regarding use of 


the system are relayed to the audit team carrying out the systems audit; 


3.2 an audit of all the RCs that use the system, but only those activities or 


structures in the organization or RC that relate to use of the system; or 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 1, Section 1 - 165 - 


3.3 an audit, similar to that described in 3.2 but of a sample of users: this 
is feasible since the audit, being an audit of a system, will necessarily 
have homogeneous elements and the main feedback path for the audit 


will be to the owner or host managers. 


Where the users of the system are outside the host organization of the internal 
audit group, the audit group will have to coordinate its activities with the 


program evaluation function. 
Pre-implementation Audit 


Pre-implementation audit is an audit of proposed legislation, policies, systems, 


contracts, etc. 


These audits are concerned generally with the degree to which the mechanism 
under design will exhibit post-implementation manageability and auditability, and 
specifically with the adequacy of controls being built into the proposed mechanism. 
They are performed at various points during the specification, development, design, 
implementation, and turnover process, when the cost-effectiveness of such a review 
is highest. A second type of review that may be included in the audit scope at this 
time is an audit of the development process itself. This latter type of audit has 


already been dealt with in the previous discussion (Systems Audit). 


Purpose: The purpose of pre-implementation audits is to audit major systems to 
assess the adequacy of controls being designed into the systems in question, prior 


to their implementation. 


Scope: The scope of pre-implementation audit includes major new or revised 
legislation, policies and procedures, information systems, program delivery systems, 
contracts, etc. As with audits of operating systems, covered in the foregoing 
discussion, systems under design involve participation of developers, owners and 
users. The perspective of the audit must take the control needs of all these parti- 


cipants into account. 
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Audit Objectives: The objectives of a pre-implementation audit are to assess the 
adequacy of controls being designed into the system from the perspective of all 
stakeholders (i.e. developers, maintainers or custodians, owners or hosts and users), 
keeping in mind materiality and significance considerations. In essence, a pre- 
implementation audit is a systems audit, as described above, except that it is 
performed before it is put into effect. Also, it does not normally include assessment 
of systems development structures or processes nor owner or user delivery systems 

- unless the system under development is the delivery system. On the other hand, 


relevant general or environmental controls usually will be assessed. 


Lines of Inquiry: The audit will generally focus on the activities of the systems 
development team since the system is a concept on paper, rather than a physical 
system. However, as indicated in the discussion of audit objectives, the control 


needs of all stakeholders will have to be assessed. 


It is not uncommon for an audit of the systems development group (organization or 
RC) to include the pre-implementation audit of one or more systems under develop- 


ment as a sub-component of the more extensive organization or RC audit. 
Special Audits 


This type of audit is usually for the purpose of reporting on an issue that does not 
fit readily into a manageable audit unit. It would include unscheduled audits 
requested by senior management, unexpected audits caused by some fraud or 


defalcation incident, etc., as well as scheduled, issue-oriented audits. 


Purpose: The special audit is usually performed at the request of management, 
normally to deal with unforeseen situations, policy developments or other senior 


management concerns, including suspected fraud or defalcation. 


Scope: The scope of a special audit depends on the situation. It is typically narrow 


and executed in a comparatively short time frame. 


Audit Objectives: The objectives of a special audit are situation dependent and 


cannot be generically stated. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 1, Section | - 167 - 


Lines of Inquiry: Since the purpose, scope and objectives are situation dependent, 
the approaches to be used cannot be generically stated. 
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SECTION TWO: AUDIT ASSIGNMENT STRATEGY 


In Section One, generic audit objectives and associated lines of inquiry were 


presented for the various types of audits that an audit group is likely to undertake. 


An analysis of the lines of inquiry will reveal that when the various types of audits 


are scheduled in a long-term plan and annual schedule there is likely to be consider- 


able duplication if the scope of individual assignments is not adjusted to remove or 


at least minimize it. 


In this section various possible strategies will be discussed for choosing the types of 
audits to perform in any one time frame and the possible adjustments in scope, 

objectives and lines of inquiry that might be made which would satisfy the require- 
ments of avoiding unnecessary duplication while ensuring adequate coverage of the 


audit universe and specific managerial concerns. 


In planning for adequate audit universe coverage, over a three- to five-year period, 


two of the first concerns are to determine: 


% which of the organization's units or elements are major and significant 


to management; and 

® what combination of types of audit (e.g. RC, program, function, system, 
special) will result in appropriate, meaningful and timely feedback to 
management. 


In this respect, some general principles may be adopted as a starting point. 


® The definition of the terms "major" and "significant" are to be arrived 


at jointly with senior management. 


* What is considered major or significant may change over time. The 


audit group must recognize this in developing audit plans. 
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e The type of audit chosen should best reflect the nature of the activity 
that the main auditee is managing (e.g. functional direction, program 


delivery) in order that useful feedback to management be optimized. 


® The type of audit chosen should respond to management's expressed 
concerns. 
® The scheduling of audit fieldwork should result in a minimum of disruption 


of the auditee's environment, in terms of both frequency of interruption 


and length of time per interruption. 


8 Presentation of reports to management must be timely, in terms of the 
currency of the data on which the findings and conclusions are based 


and in terms of management's decision cycles. 


8 The choice of audit strategy should result in efficient use of audit 
resources (i.e. the use of audit resources for results achieved should be 


optimized). 
Definition of Major and Significant 


In the Standards’, the terms major and significant are used in the context of "All 
major systems, functions and organizational units performing significant 
responsibilities should be examined within a period not exceeding three to five 


years." 


Since the purpose of internal audit is to provide advice to management, clearly the 
definition of what is major and significant must be determined in the light of manage- 
ment's perception of the relative importance of its various organizational elements 


and associated responsibilities. 


7 Standard No. 3, Standards for Internal Audit in the Government of Canada, 


Treasury Board of Canada, (Office of the Comptroller General), 1982. 
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The term major is usually dependent on significance and significance is usually 
determined in terms of risk of deviation in actual as opposed to desired performance 
which may result in a material or significant loss of resources or unacceptable 


program output or effects. 


Although, ideally, management would like reassurance that all its organizational 
elements are performing satisfactorily and that risk of loss is minimized, resources 
are limited. Therefore, there are some organizational elements for which control 
activity (managerial contro! and/or internal audit) will not be cost-justifiable. For 
this reason what is considered major or significant will depend on a trade-off 


between the risk of loss or non-performance and cost of controls. 


As organizational objectives and priorities change, the definition of major 
organizational units and significant responsibilities will necessarily change 


accordingly, while still subject to resource constraints. 
Choice of Audit Types 


In deciding which type of audit will best provide the desired audit universe coverage, 
"optimization of advice to management" may again be used as a criterion. The 

type of audit chosen should be such that it answers all of the auditee manager's 
significant questions in the most efficient manner. Efficiency, then, is the second 
criterion. Efficiency in this case has two aspects: assignment level efficiency and 


overall operations level efficiency. 


The interaction of the "advice to management" and "efficiency" criteria may best 


be illustrated by the following example. 


All organizations have staff (functional) and line (program, product, service 


delivery) organizational units. 


Suppose we are planning to audit one functional unit, say finance, and several 


program delivery responsibility centres. 
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The preferred audit type for auditing a functional organization unit is a function 
audit. This type of audit typically includes an organization or RC audit of the 
organization's units that provide the functional advice, direction, services and an 
audit of a sample, or all, of the organizational units or RCs that are subject to the 


advice, direction and services provided. 


If the function audit is scheduled and performed without regard for any line or 
program organizational unit audits that may be planned for the same time frame, it 
is likely that many organization units will either be visited twice in the same 
general time frame (say a year) or, worse still, simultaneously by two different 
audit teams. Thus it may be readily seen that an approach which maximizes advice 
to the functional manager may result in inefficient use of audit and auditee 
resources. Clearly a trade-off is called for, one of optimization of the two 


criteria rather than maximization of either. 


An alternative strategy is to perform the organization or RC audits for both 
functional and program managers independently but with provision for sharing 
relevant results both ways. That is, results of program-based audits that involve 
findings pertaining to advice, direction and services provided by the functional 

group would be passed on to the function audit team while the results of those 
program delivery organizational units that were in the function audit sample but 
were not planned for audit in the same time frame would be passed on to the program 


audit team. 


This may involve altering the scope of some of the audits in order to accommodate 
the needs of both teams. However, the result is that the "advice to management" 
and "efficiency" criteria are optimized; i.e. the best results possible are provided 
for management while meeting cost constraints and, incidentally, minimizing 


annoyances to management which may or may not have cost implications. 


There are two main limitations to this strategy. The first is currency of data. 
That is, results from one audit will only be useable by another audit team if the 
data upon which the audit findings are based are still valid. The second is audit 


perspective. Although the scope of a program audit may be changed to provide 
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needed feedback to a function audit team, the perspective will still be, or should 
be, that of the program manager and that of the functional manager, for the reverse 


situation. 


This is possibly less of a limitation than it seems. Often a program manager goes 
overboard in depreciating the importance of central direction while the reverse is 
also true: i.e. functional managers often overemphasize the importance of functional 


direction. 


The auditor is in an ideal position to balance the relative importance of both, thus 
providing a more balanced opinion from the point of view of the head of the whole 


organization - the ultimate client, the Deputy Head. 
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CONCLUSION 


In conclusion, a natural extension of the audit purpose and scope statements for 
audit assignments, discussed in Part 1, is detailed audit objectives and associated 


lines of inquiry. 


As is the case with purpose and scope, audit objectives and lines of inquiry are 
determined by the type of audit being undertaken and the nature and scope of the 
audit unit being audited. Beyond this, audit objectives and lines of inquiry may also 


be influenced by specific management concerns. 


In Section One of this chapter, general audit objectives and lines of inquiry have 
been presented for a number of the more typical types of audits that an audit group 
may undertake. In each case, subject- and situation-specific adjustments would 
have to be made in order to have a complete set of either audit objectives or lines 


of inquiry for any specific audit assignment. 


In Section Two, audit assignment strategy was discussed, some general principles 
which may be useful in deciding on such strategy were presented and criteria for 
choosing between audit types were enumerated. In particular, the trade-off between 


maximizing advice to management and effiency was stressed. 


In general, this chapter supplements the discussion of the audit assignment process 
introduced in Part | and specifically rounds out the areas related to subject- and 


situation-specific aspects of audit assignment planning and execution. 
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CHAPTER TWO 
CONTROL: CONCEPTS AND APPLICATIONS FOR INTERNAL AUDITORS 
INTRODUCTION 


Public concern over the size, complexity and cost of government activities has 

placed increased emphasis upon the importance of adequate managerial! control in 

the federal government. The importance of proper control derives from the public 
service's obligation to render to Parliament a full accounting for the carrying out 

of its assigned responsibilities and for its stewardship of the resources entrusted to 

it. The ability of public sector managers to account for their activities requires 

that they have a means of directing, monitoring and redirecting performance in 

each of their delegated areas. To this end, managers must establish adequate systems 
of control. 


The recent change in the nature, scope and purpose of internal auditing in the federal 
government places the audit profession in a unique position for providing aid to 
management in achieving improved control. Internal auditors are currently expected 
to possess a fundamental understanding of control theory and be capable of 
commenting on its proper application to organizational settings. The Standards for 
Internal Audit indicate that the role of federal government internal auditors is "to 
review, evaluate and report on the adequacy of the ...control framework... to 
examine and evaluate performance in relation to this framework and to report the 
extent of compliance to management". The Institute of Internal Auditors has stated 
that "internal auditing is to assist members of the organization in the effective 
discharge of their responsibilities by providing them with information regarding 
control".! Internal audit as the "control of controls" is considered to be an essential 
tool of management, providing a service through its independent review and 


evaluation of the economy, efficiency and effectiveness of other controls. 


1 The Institute of Internal Auditors, Statement on Internal Auditing Standards 


No. !, Control: Concepts and Responsibilities, Institute of Internal Auditors, 
1984. 
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Despite the significance of the concept of control to the federal government in 
general and the internal audit community in particular, there still exists considerable 
ambiguity in the actual use of the term control. This may be the result of 
inadequate thought about control in conceptual terms. Perceptions relating to 
control are often restricted to the applied aspects of the subject with attention 
centred upon the various physical devices, or process steps, considered necessary to 
achieve control. Too seldom is control discussed in terms which define its funda- 
mental purposes, elements, functions and relationships to the entity subject to 


control. 


Recent publications, such as Principles for the Management of the Public Service 
of Canada (see Bibliography at the end of this chapter), move managers and internal 


auditors towards a clearer definition of control; however a greater in-depth under- 
standing of the subject is still required. Managers require a basic understanding of 
the subject to ensure that an appropriate design and application of controls occurs 
within the government. Internal auditors must possess a proper conceptual under- 
standing of control if they are to effectively fulfil their redefined role as the 
organization's control advisers. Meaningful and valid audit results will be 
consistently achieved only when broad-scope auditing is fully reconciled to the 


valid conceptual basis offered by control theory. 


In addressing the need for a valid conceptual basis, this document discusses systems 
and control theories both in general terms and as they are applied to audit assign- 
ments. In Section One, an overview of the internal audit function is given with a 
demonstration of where systems modelling techniques and control theory can con- 


tribute to the conduct of the audit assignment. 


Having shown the usefulness of systems and control concepts, Section Two focuses 
on the core knowledge auditors should have relating to these subjects. The reader 
should recognize that this section involves discussion of a number of abstract ideas 
which initially may be difficult to follow because they do not specifically address 


the applied aspects of auditing. 
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The significance of these generic concepts will become apparent in Section Three. 
Here, the document shows how organizational activities can be modelled, and the 
auditor's evaluative criteria developed, using the systems and contro! concepts 
developed in Section Two. Section Four demonstrates how the descriptive and 
prescriptive models are then used by the auditor to evaluate controls. Included in 
this discussion is the integration of the suggested audit approach provided in this 
document with the conventional work instruments associated with audit assignments. 
Specifically, a brief reconciliation is given between the use of the Internal Control 
Questionnaire, the Audit Verification Program and the prescribed audit approach. 

It is hoped that this structure illustrates not only a practical approach to the conduct 
of audits, but also represents a methodology which is based on a firm conceptual 


understanding of control. 


Management is responsible for the identification of the need for and the 
development, implementation and operation of the control framework. Accordingly, 
Chapter 3, "Management Control: Concepts and Practices", deals with these aspects 


of control and should be read in conjunction with this chapter. 
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SECTION ONE: INTERNAL AUDIT AND CONTROL THEORY 

The Controls Approach to Internal Audit 

In general, internal auditing in the federal government is a function whose purpose 
is to arrive at certain conclusions concerning the condition of the resources, 
processes and results of the entity under review. Specifically, the Standards for 


Internal Audit expect the auditor to determine whether: 


® systems, procedures and controls are adequately designed and developed, 


and that they are operating efficiently and effectively; 


e adequate information is available for decision-making and accountability 


purposes; 

9 available information is properly utilized in the decision-making process; 
e public funds and assets are adequately protected; and 

a legislative, central agency and departmental directives are being 


complied with. 


Internal auditors usually do not attempt to assess directly the operations and results 
of the entity under review to determine whether the above conditions exist; rather 
they evaluate the adequacy of the control framework established by management 

to achieve these conditions. Essentially, the control framework serves as a proxy 
measure for determining whether the desired conditions actually exist. Where the 
auditor determines that the control framework is effective, the underlying inference 
is that the entity under review likely achieves satisfactory operating conditions. 

For example, consider the auditor's task of concluding whether the level of efficiency 
found in a system under review is adequate. The auditor's tests of the control 
framework will not give a full appreciation of the system's actual efficiency but 


will give assurance that management is doing everything practical to ensure that 
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desired levels of efficiency are achieved. Using tests of the control system, the 
auditor can conclude that there is reasonable assurance that the operations are 


achieving an adequate level of efficiency. 


To evaluate the adequacy of the auditee's control framework, the auditor must 
perform two tasks. First, the auditor must develop a sound understanding of the 
actual processes and controls subject to review. Second, criteria must be determined 


which will form the basis of the auditor's evaluation. 


In essence, auditing involves the comparison of the actual conditions of the auditee 
operations as determined by the auditor with the evaluative criteria that have been 
established for the assignment. The audit report should identify those areas where 
actual auditee conditions either conform or deviate from the desired conditions 


represented by the evaluative criteria. 


While simple in conceptual terms, the actual development of an accurate under- 
standing of the auditee's operations and the creation of evaluative criteria are 
complex tasks. The major portion of the balance of this document is directed at 


providing the theory and techniques required by auditors to complete these tasks. 
The Development of Audit Models 


Models are of particular importance to analysts of all types of organizations. Faced 
with a confusing array of facts, analysts use models to bring order to their under- 
standing of the entity under review. In auditing, models are useful not only in 
bringing order to the auditors' study of the auditee, but also as a means for 
communicating their understanding and basis of evaluation to other auditors, auditees 


and audit report recipients. 


The theory and techniques that an auditor can use to develop both a descriptive 
model of the entity under review ("what is") and a prescriptive model containing 


the audit assignment's basis of evaluation ("what should be") is explored here. 


In the development of a descriptive model, this document emphasizes the usefulness 
of systems modelling techniques and control theory in characterizing the auditee's 


key structures, processes and controls. 
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The prescriptive audit model sets out the basis on which an auditor will evaluate 
the adequacy of the auditee's controls. As described, the prescriptive model must 
at least establish the auditor's expectations in terms of what controls are essential, 
what objectives they should serve and how the controls should be designed. The 
development of this model must also rely heavily on a sound knowledge of control 


theory. 


Summary 


A sound knowledge of control theory is essential to auditors as it provides the basis 
for their evaluation of the adequacy of the auditee's operations. An orderly 
evaluation of complex auditee operations is facilitated through the auditor's use of 
descriptive and prescriptive models which are based on systems modelling techniques 
incorporating control theory principles. The balance of this chapter is devoted to 
developing the auditor's understanding of systems techniques and control theory so 
that the above approach to audit evaluation can be achieved. Section Two will 
provide the basic knowledge requirements of systems modelling and control theory; 
Section Three will then demonstrate the application of these techniques and theory 


in the audit process. 
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SECTION TWO: SYSTEMS MODELLING AND CONTROL THEORY 


This section of the chapter is divided into two parts. In the first part, discussion 
centres on systems theory and modelling techniques which should help auditors 

build descriptive models of the entity subject to audit. The second part reviews 
basic control theory principles borrowed from the physical sciences and cybernetics. 
Employing the concepts of systems modelling, control is presented as a particular 
type of system having a unique kind of purpose and structure. This information 
provides the basis of building control features into the auditor's descriptive model 
of the audited entity. An understanding of the purpose of controls and how they 
should be designed will also form the core knowledge that an auditor should employ 
when building the prescriptive model used to evaluate the adequacy of the auditee's 


operations. 


The discussion in this section is conceptual; demonstration of the usefulness of the 
material covered here is reserved for Section Three. Throughout the text of 
Section Two, summary statements relating to systems and control are highlighted 
for the reader's attention. These statements provide in concise form the key 


concepts that form the basis for the control model put forward. 


Systems Concepts 


What is a System? 


A system is defined as a set of elements related to one another according to some 
coherent pattern. While the elements are important, it is the linkages or relation- 
ships among the elements, defined in terms of a common purpose, which make it 
possible to speak of a system. The elements of a system and their relationships 
have attributes or properties such as size, shape, function, velocity or rate. It is 
the relationships however that account for the systemic nature of phenomena rather 


than the attributes themeclyecs 


2 White, Michael J., et al, Managing Public Systems: Analytic Techniques for 
Public Administration, North Scituate: Duxbury Press, 1980, p. 21. 
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Basic to systems is the idea of purpose. Coherent patterning associated with the 
elements of a system implies the existence of meaningful relationships. Relation- 
ships become meaningful when linked with a common purpose. Without knowledge 
of purpose, an observer would not be able to distinguish the elements and relation- 
ships relevant to a system; the boundaries of the system could not be determined 


and there would be no awareness of Structure:s 


Elements included within a system can form relationships of varying strength. The 
strength of relatedness between elements is determined by the degree to which 
their interaction contributes to the achievement of the system's overall purpose. 
Many variables interact to contribute to the attainment of a particular system's 
purpose. The model builder, however, makes conscious choices as to which relation- 
ships are most significant in explaining the process through which a purpose is 
achieved. Proper selection of system elements and relationships in a model allows 
for a useful representation of reality while avoiding the costs associated with 


unnecessary detail. 


® Summary Statement: Entities which exhibit purpose can be described 
using systems techniques. 


Basic System Features’ 


The basic system features can be represented as shown in Figure 1. Although 
specification of a system includes elements, relationships and their respective 


attributes, we need several more descriptive concepts. 


The first thing one should notice about the systems diagram is the introduction of 
the terms input, process, output and feedback. Inputs are the start-up force that 


provides the system with its operating necessities, be they demands for products 


3 Beer, Stafford, The Heart of Enterprise, Chichester, New York, Brisbane and 
Toronto: John Wiley & Sons, 1979, pp. 7-9. 


4 Schoderbek, Peter P., Kefalas, Asterios G., and Schoderbek, Charles G., 


Management Systems Conceptual Considerations, Dallas: Business Publications, 
Inc. 1975, pp. 30-37. 
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SYSTEM CONCEPTS* 


SYSTEM’S ENVIRONMENT 


SYSTEM’S BOUNDARY 


PROCESS 


FEEDBACK 


Figure | 


Schoderbek, Peter P., Kefalas, Asterios G., and Schoderbek, Charles G. 


4 
Management Systems Conceptual Considerations, Dalias: Business Publications, 


Inc. 1975, pp. 30-37. 
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and services or resources such as material, energy, humans or information. Inputs 
can be the outputs of other systems or be a reintroduction of a portion of the output 
of the same system. This latter type of input, called feedback, will be discussed 


shortly. 


Process is that which transforms the input to an output. In some situations, the 
process by which inputs are transformed into outputs is not known in detail because 
this transformation is too complex. Observers of such systems can only make 


inferences about the nature of the input-to-output conversion process. 


Outputs, like inputs, generally take the form of products, services, information or 
energy. The output of one system becomes the input of another system. The 
succeeding system may represent a new cycle of the process which just created the 


output or may be a new process. 


Feedback is a systems component which ensures that the desired state of the system 
is maintained or attained. Without feedback, a system would not have the infor- 
mation to determine whether actual performance compares favourably with desired 
ends or whether the means to desired ends, or process, will continue to perform as 
expected. Feedback represents the control function of a system and establishes 


the basis from which our control model will be subsequently derived. 


Each of these system concepts will be re-examined in Section Three. The nature of 
inputs, processes, outputs, and feedback in organizations will be reviewed to provide 
auditors with a basis for creating a descriptive model of the processes subject to 
audit. 


Environment of a System 


One should also notice that Figure | identifies a system's boundary and a system's 
environment. Every system has something internal and external to it. An environ- 
ment is external to a system but affects the system's behaviour. The behaviour of 


the environment can be influenced by the system, but cannot be controlled by it. 
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A method proposed for the determination of a system's environment is to pose 
certain questions as illustrated in Figure 2. The first question is whether the item 
under observation is relevant given the system's objectives. If the item is not 
relevant to the system being modelled, then it is neither included in the descriptions 
of the system nor its environment. If the item is relevant, however, then the second 
question is asked to determine whether the item is also subject to the control of 

the system. Where the item is controllable it is systemic, where it is uncontrollable 


it is environmental. 
IDENTIFICATION OF SYSTEMIC OR ENVIRONMENTAL FACTORS? 


Is the item under observation 
relevant to the system's objectives? 


Item is included Item is neither 

as part of the part of the system 
Is the item under system nor its environment 
observation controllable (e.g. not included 
by the system? Item is included in systems model) 


as part of the 
system's 
environment 


Figure 2 


As will be discussed in Section Three, one of the strengths associated with the 
systems approach is its direct recognition of environmental factors which influence 
system's behaviour. Understanding environmental influences will often provide 


clues as to the causes behind certain aspects of the system's behaviour. 


For auditors, specific recognition of environmental influences is an important aspect 
of their analysis. In many cases, auditee actions represent responses to environ- 
mental influences. Any audit model which does not consider the auditee's operations 
within the context of its larger environment risks losing a significant portion of its 
power to explain auditee behaviour. 


—_—— 


2 Schoderbek et al, op. cit., pp. 39-41. 
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Boundary Problems 


The boundary of a system depends upon the observer's viewpoint as to the nature 

and purpose of the system. Once these have been specified, the facts that are 
relevant to the observer's conceptualization become fixed and a boundary separating 
controllable from uncontrollable factors can be drawn. : 

When a number of observers are analyzing a system, the boundary may be established 
through conventional agreement. Otherwise, given the possible differing perspectives 
of the observers as to the nature and purpose of various phenomena, there is a 


danger that no common systemic framework will be derived.© 


Determining boundaries is an important aspect of the auditor's descriptive model as 
they establish which activities should be subject to the auditor's direct examination. 
Differences in opinion between the auditor and auditee as to which activities or 
elements are controllable and relevant to the auditee's operations should be subject 
to particular scrutiny. In many audits, these differences in opinion may reveal 
important problems concerning the auditee's perception of jurisdictional boundaries 


within the organization. 


Levels of Systems’ 


The input - process - output diagram of Figure | may represent a model of a whole 
system or only part of a larger system. When clusters of elements within a system 
are related by a common purpose they comprise a sub-system which follow some 


or all of the same norms of behaviour as found in the larger system. 


Having identified a collection of elements, relationships and sub-systems as a system, 
the system's environment can be examined. It may be found that the system under 
investigation can be viewed itself as but a component of a larger collection of 


elements and relationships sharing common purposes and affected in common by 


6 Beer, op. cit., pp. 9-10. 


7 White et al, op. cit., pp. 26-27. 
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each other and wider events. This wider collection of units and relationships that 
form part of the system's environment may be termed for convenience as the supra- 


system. 


The terms sub-system, system and supra-system are applied according to the 
observer's (or auditor's) level of interest at a particular moment. These terms are 


illustrated in Figure 3. 


Open versus Closed Systems 


The classification of systems into open and closed categories rests upon the concepts of 
boundaries and inputs. Resource inputs over which categories the system has control 
are, by previous discussion, within the boundary of the system. Uncontrolled inputs 


are part of the environment.® 


In an open system, uncontrolled inputs are accepted by the system and may influence 
its future behaviour. In a closed system, uncontrolled inputs are not accepted and as 


such cannot influence the systems behaviour. 


For the auditor, knowing whether a system tends to be more open or closed may 
help in determining whether a system is properly designed and operated to accom- 
modate the demands of its environment. Where a more closed system exists ina 
rapidly changing environment, the auditor may be concerned about the continuing 
effectiveness of the system. With a lessened ability to accept new environmental 
information, there is a danger that the system will not respond appropriately to 
changing environmental needs and constraints. Contrastingly, where a system is 
overly receptive to environmental information which has limited bearing upon the 


system's effectiveness, the auditor may be concerned about the system's efficiency. 


Adaptive versus Corrective Systems 


Adaptability is a system's ability to alter its processes so as to produce the kinds of 
outputs required for continued survival in its environment. The learning capability 


is the key characteristic of adaptive systems. They may respond to requirements 


8 Schoderbek et al, op. cit., pp. 45-46. 
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SUPRASYSTEM, SYSTEM AND SUBSYSTEM 


SUPRASYSTEM A 


SYSTEM 
Aj 


SUBSYSTEM SUBSYSTEM 
A11 A12 


Figure 3 
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for changed outputs in unprogrammed or non-routine ways. Adaptive systems must 
be open. Identification of a need to alter systemic behaviour to produce desired 


outputs is dependent upon receipt of environmental information.” 


The nature of adaptive systems can be compared and contrasted against the charac- 
teristics of corrective systems. Like adaptive systems, a corrective system may be 
open to environmental information and be capable of detecting changes in it. 
Corrective systems can also adjust their processes and outputs in response to the 
detected environmental changes. Unlike adaptive systems however, the corrective 
system has no learning capability. Corrective systems can only respond according 


to predetermined rules and do not have the flexibility to adjust those rules. 


Auditors are concerned with the degree of flexibility with which a system can react 
to changed environmental conditions. In certain situations, where the environment 

is relatively unchanging or changing in predictable ways, it may be more efficient 

to program a system to respond in a fixed manner to certain environmental conditions. 
On the other hand, where an environment is constantly changing, the system may 

need to be very adaptive to remain effective and responsive to environmental 


requirements. 
Summary 


Systems theory is often seen as a strategy for inquiry, a way of thinking. Asa 

model, systems theory provides a way of seeing. In the following part, we will be 
developing a model of control using systems theory and incorporating the concepts 
presented in this section from the specific perspective of control systems. As will 

be shown in Section Three, the descriptive model of the auditee developed by auditors 


may be designed using control system modelling techniques. 
Control Systems 


The word control has been used with such a variety of meanings in audit and 
management literature that it has achieved a high level of ambiguity. Control is 
often described in terms of its manifestation as a physical device (i.e. wage and 


——_—— 


9 Schoderbek et al, op. cit., pp. 47-48. 
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price controls), a step in a process (i.e. regulating, directing), or as a particular end 
result and condition (i.e. "in control", "under control"). While some attention is 
given to the purpose of these devices and procedures, a rigorous treatment of the 


concept of control is not available. 


One of the reasons for the focusing of attention on control applications is due to 
the dominating influence of private sector external audit literature and research on 
the audit community as a whole. In external auditing, evaluation of financial 
accounting controls is performed to determine whether the accounting system can 
be relied upon to generate financial data in accordance with generally accepted 
accounting principles. Where reliance on the system of internal control can be 
justified, the auditor may reduce the extent of testing that must be performed to 
substantiate any opinions expressed on the fair presentation of financial reports. 
Evaluation of financial accounting control has been aided by the development of a 
number of generally acceptable control practices which can be used as a template 
by auditors during their evaluation. Since standard practices exist, attention paid 
to controls tends to be oriented towards the application side of the accounting 


control model as opposed to the underlying concepts involved. 


In the practice of management and in internal auditing, controls are examined ina 
much broader context than strictly financial accounting control. While some of the 
financial control principles used in external audit evaluations are helpful to internal 
auditors, additional control criteria are required to meet the internal auditor's 
needs in evaluating managerial and organizational activities. These criteria must 
be even more extensive for public sector internal auditors who are concerned with 
the adequacy of controls over not only the economic results of the organization but 
social results as well. Instead of focusing on the numerous control applications 

that can be found in the public sector, internal auditors must strive to first obtain 
a fundamental understanding of the purposes of control and the elements and 
activities necessary to achieve these purposes. Once such a conceptual under- 
standing is achieved, public sector auditors will then have a generic basis from 
which all the wide-ranging manifestations of control can be identified and evaluated. 
This comprehensive understanding of controls is a fundamental element of a 
professional auditor's body of knowledge and the basis for the auditor's credibility 


with auditee management. 
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In this part, the concept of control is explored using the systems theory introduced 
in the preceding part and contro! principles borrowed from the physical sciences 


and cybernetics. 


Control as a Particular Type of System 


In the previous part dealing with systems theory, the concept of feedback was 
introduced. We noted that feedback is a necessary component in any system which 
is designed to ensure that a certain desired state is attained or maintained. Feedback 


then represents the control function of such systems. 


Here, we will model the feedback control function as a system represented by a 
number of elements and activities purposively related. As such, we are viewing 
feedback or control as a sub-system within the boundaries of our system developed 


in the previous part. 


Control systems are described here in general terms without specific reference to 
their application in audits. The task of applying control theory to the audit context 
is reserved for Section Three which builds an organizational model using the systems 
techniques of the preceding part and control concepts provided here. The approach 
of moving from general concepts to specific applications is intended to ensure that 
a common conceptual understanding of control exists before considering how these 


concepts are applied to the particular requirements of the auditor. 


To develop a systems model of control, we will begin with a statement regarding 
the purposes of control. With such a statement, we can then deduce the boundaries 


of the system, its elements, relationships and activities. 
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The Purpose of Control Systems 


Various authors have suggested the following definitions relating to the purpose of 


control systems: 


"A control system is a system whose purpose is to attain and maintain a desired 


state or condition." (Anthony and Dearden)!? 


"We are concerned with control in relation to matching performance with 


necessary or required conditions to obtain a purpose or objective." (Litterer)!! 


"Control is concerned not only with the events directly related to the accom- 

plishment of major purposes, but also maintaining the organization ina 

condition in which it can function adequately to achieve these major purposes." 
; 11 

(Litterer) 


"The problem of control is that of obtaining a desired result in the face of 


conditions that might oppose or interfere." (Rubenstein and Haberstroh)!4 


While stated somewhat differently, these samplings from management literature 
present a consistent picture of the purpose of control systems. Essentially, control 
systems are used to ensure that the purposes or objectives of the system under 
control are attained or maintained. Control systems are only relevant and are only 


applied to their parent systems, which themselves have an identifiable purpose. 


Control systems help underlying systems attain or maintain their purpose by doing 
two things. First, control systems regulate the output of the systems being controlled 


by ensuring that actual results compare favourably with desired results. Second, 


10 Anthony, Robert and Dearden, John, Management Control Systems, Homewood, 
Illinois: Richard D. Irwin, Inc. 1980, p. 3. 


11 Litterer, Joseph A., The Analysis of Organizations, New York: John Wiley & 
Sons, Inc. 1965, p. 233. 


12. Rubenstein, Albert H. and Haberstroh, Chadwick J., Some Theories of 


Organization, Homewood, Illinois: Richard D. Irwin, Inc. and the Dorsey 
Press 1966, p. 503. 
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control is concerned with means as well as ends. To ensure that the system under 
control obtains its purpose, control systems must also be involved in maintaining 
the internal capability of the controlled system such that it is able to continue to 


produce the desired outputs. 


If the influences on the system being controlled, and the controlled system itself, 
were unchanging or changing in predictable ways, the need for controls would be 
minimal because there would be little risk that a process, once set in motion, would 
not achieve the expected results or its ultimate purpose. Environmental influences, 
however, rarely remain stable over time and the controlled system itself is subject 
to changes (e.g. wearing out). It follows that if the purposes associated with such 
systems are to be maintained or attained then it is necessary to impose some sort 
of control system so that any tendency towards instability may be detected and 


corrected. 


# Summary Statements: The concept of control is only relevant for 


purposive entities. 


Controls exist to ensure that systems attain or maintain certain desired 


states or conditions. 


Model of a Control System: The Basic Plemente.? 


Figure 4 presents a model of a control system. While this model represents a 
particular type of control system, it is useful for identifying the basic elements, 


activities and relationships associated with all control systems. 


Once we have introduced these basic aspects of control systems, we will indicate 
how the control structure may be altered to accommodate the specific needs of the 


system subject to control. 


13. Adapted from Schoderbek et al, op. cit., p. 308-310. 
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The Control Object 


The control object is the variable of the system's behaviour chosen for monitoring. 
Changes in the status of the control object trigger the functioning of the control 
system. Obviously the usefulness of the control system is therefore dependent 


upon the proper choice of the control object. 


In all control systems, some of the control objects must be chosen from the 
controlled system's outputs. A system seeks to attain certain results. Without 
some control over the variations in the state of the system's outputs there would be 
no information regarding whether actual performance was achieving desired ends 
or whether adjustment was required. Additional control objects, however, can be 
located over the input variables and conversion processes of the system under 
control. Input controls are used to either prevent faulty input from entering the 
system or to signal the need for adjustments in the conversion process. Conversion 
controls ensure that the conversion process is working as intended. Figure 4 is an 
example of a control system where output is the subject of control activity. 


Figure 5 illustrates a control system with an input as a control object. 
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13 


Adapted from Schoderbek et al, op. cit., p. 308-310. 
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1. Activator is shown to have two alternative actions; one, an adjustment to processes yet to be 
performed; two, an adjustment to previous processes. 


Figure 5 
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Detector 


The detector measures what is happening in the variable being controlled. It 
represents the control system's information supplier. As such, many of the concerns 
associated with information systems in general could be applied to the design and 
evaluation of a detector. Consideration, for example, as to frequency, capacity, 
efficiency, precision, cost, representativeness and adequacy of the detector as an 


information system should be given during design and evaluation. 
Reference Point 


This represents the standards against which performance may be measured or 
matched. Typical concerns in this area relate to the completeness and appropriate- 
ness of the standards in establishing performance criteria for the various dimensions 


of the systems behaviour including quality, quantity, timing and cost. 
Comparator (Analyzer) 


This element represents a device for assessing the significance of what is happening, 
usually by comparing information supplied by the detector (what is actually 


happening) with the established reference points (what should be happening). 


This mechanism then establishes the nature of the difference between actual 
performance and the standard, and passes its judgment of whether the system is 


operating as intended on to the next element. 
Activator 


The activator is a decision-maker. It evaluates alternative courses of corrective 
action available given the nature of the deviation identified and transmitted by the 


comparator. The output of the activating mechanism is typically corrective action. 


Corrective actions are usually directed at adjusting the operating processes of the 
system under control. The output of the activating system, however, can also be a 


corrective action aimed at altering the control sub-system itself. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 2, Section 2 - 198 - 


In a closed system, we have already noted that there is nothing unpredictable about 
a system's response; it is programmed to perform according to specification. As 
such, in a closed system the activator would deal only with situations for which 
there is a decision rule for the deviation detected. We will see however in the 
discussion of open control systems that the "no decision rule" state can exist and is 


significant. 
Summary 


The foregoing basic elements should be found in all controls. Whenever auditors 
describe controls, specific recognition to each of the elements and their functions 
should be given to allow for proper evaluation of their adequacy. In Section Three, 
each of these generic control elements ‘is redefined within an organizational context 


and its use by auditors specifically described. 


e Summary Statements: In all control systems some of the control 
objects must be chosen from the controlled system's outputs. 
Additional controls may be located over the input variables and 


conversion processes of the system under control. 


All controis contain the elements of a detector, reference points, a 


comparator and an activator. 


Alternative Design Strategies for Control Systems 


To this point, we have emphasized the purposes, elements and activities thought to 


be associated with the basic nature of control systems. 


As we become more precise about the nature and purpose of the system to be 
controlled and more demanding of the type of control information that needs to be 
generated to achieve our control purposes, certain distinctions can be made in the 
detailed characteristics associated with various control systems. These differences 


are explored in the following. The purpose is to provide additional concepts to 
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auditors so that they may more accurately describe the characteristics of a 
particular control. Recognition of various types of controls and where they are 
most effective will also be useful to auditors in building their prescriptive model 


which forms the basis for evaluating the auditee. 


@ Summary Statement: The specific design of controls should vary in 
accordance with the nature of the system under control. 


Strategies for Achieving Control Purposes! * 


At the highest level, three strategies for achieving control purposes can be identified. 


These strategies are: 


® Yes/No controls - system activities may not proceed to the next step 
until a screening test has been passed; approval to continue is required; 


(control is preemptive). 


® Steering controls - results are predicted and corrective action taken 


before the system's processes are completed; (control is preventative). 


® Post-action controls - system activities are first completed; results are 
then measured and compared with a standard before control action is 
taken; control action is directed towards eliminating the deviation in 


future cycles of the process under control; (control is detective). 


Each strategy represents a different viewpoint as to the type of interaction required 
between a system and its controlling sub-system. All strategies represent feasible 


ways of achieving the generic control purposes previously described. 


14 Newman, William H., Summer, Charles E. and Quirby, Warran, The Process of 
Management - Concepts, Behaviour and Practice, Inglewood Cliff: Prentice 
Hall 1972, pp. 455-456. 
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Yes/No controls are essentially safety devices. The consequences of a faulty 
aspect within the system under control are considered so serious that precautions 
are taken to ensure that such fault does not occur. For example, the precautions 
taken to ensure that a parachute is not faulty may be thought of as Yes/No 
controls which would preempt use of the parachute if it did not meet adequate 


standards of quality. 


Steering controls are preventative in nature and meant to provide remedial action 
while results are still in the process of being achieved. Results are predicted ata 
particular point in the system's processes and where necessary corrective action is 
applied to ensure that actual results come as close as possible to desired results. 
For example, the maintenance function may be viewed as a type of steering control. 
This activity seeks to direct attention towards weaknesses in the existing system so 
as to prevent trouble from arising. It is emphasized here that the primary aim of 


the activity is to direct attention rather than to evaluate. 


Post-action controls are evaluative mechanisms which can adjust or provide planning 
data for future cycles of the process being controlled. These controls can also 
serve as a means of determining rewards for performance. Budgetary control and 


performance appraisals are types of this control strategy. 


The adoption of a particular strategy for control will dictate that certain 
distinguishing characteristics can be associated with the various control systems. 

A Yes/No control is preemptive because its defining characteristic is that it may 
preclude further activity depending on the results of a screening test. Steering 
controls attempt to sustain an acceptable type of performance. These controls are 
forward-looking and preventative in nature, taking action to avoid the occurrence 

of undesirable behaviour. Post-action controls are reflective and restore performance 
to acceptable standards. Such controls are detective, taking action on the basis of 


identifying deviations that have already occurred. 


Auditors must understand the purposes of the various control strategies if they are 
going to evaluate properly controls for effectiveness. Different operating systems 
require different controls and the foregoing general types provide one level of 


distinction. 
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® Summary Statement: Controls may be preemptive, preventative, or 
detective. 


Environmental Considerations (Open and Closed Control Systems) 


As noted earlier, a system's environment is outside of the system's direct control, 
but relevant portions of the environment are significant in that they include needs 


and constraints which influence and can be influenced by the system's behaviour. 


The complexity and predictability of environmental influences upon a system should 
be reflected by the degree to which a control system is open or closed. Where the 
influences of the environment are reasonably simple to identify and have a predict- 
able effect upon a system, or where the custodian of the operating system decides 
not to respond to the environment (e.g. monopoly situation), the control sub-system 
tends towards being closed. The system's behaviour is predetermined and uses a 
limited set of environmental information. The control sub-system is essentially 


impervious to any additional input from the environment. 


Where relevant environmental factors are complex and unpredictable, a greater 
degree of openness should be exhibited by the control sub-system. Open systems 
accept additional information from the environment and therefore increase the 
system's ability to adjust its behaviour in a manner which satisfies environmental 
needs and constraints. In public organizations, most control sub-systems are open 


and the important consideration involves the question of degree of openness. 


e Summary Statement: Controls may be open or closed to uncontrollable 
(environmental) input. Where simple, predictable environmental 
influences exist, controls should tend towards being closed. Otherwise, 
controls should tend towards being open. 


Levels of Control Systems 
In our general systems model, we introduced the idea of different levels of systems 


depending upon the observer's perspective of scale. Based on the nature of the 


phenomena being studied, various levels of resolution (supra-system, system, 
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sub-system) could be adopted. It follows that as the system, and consequently the 
object of control, varies in terms of scale, so also would the scale of its corres- 


ponding control sub-system. 


Besides levels of scale, controls can also be classified according to levels of 
complexity. One way of viewing levels of contro! according to degree of complexity 
is based upon the degree of the underlying system's complexity. The more complex 
the system, the more complex the corresponding control sub-system must be. 
Complexity in this sense means a system which has many components and elaborate 
interrelationships. All elements of the control sub-system must have sufficient 
richness in complexity if it is to detect and control the variety of possible states 


that can be exhibited by the object of control. 


Control systems can also, however, be ordered according to the complexity of the 
functions they perform. Consider the following hierarchy of controls based upon 


complexity of the functions performed: 


First-order Control Systems (Corrective) 


- the object of control is monitored against a predetermined goal; 

- the control system is given particular commands to carry out, 
regardless of changes in the environment; and 

- there is nothing unpredictable about the systems response, the 


control is programmed to perform according to specification. 


Second-order Control Systems (Adaptive) 


- can perform all first-order functions; 

- the control can initiate alternative courses of action in response 
to changed external conditions; 

~ the control has the ability to change standards or decision rules 
which dictate lower-order control system behaviour; and 

- in extreme cases the control system has the capability to trigger 
the redesign of the operating system, the control sub-system, or 
both. 
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The adaptive control system is a higher-order system in that it can perform more 


complex functions than the corrective system. 


Figure 4 illustrates a closed, corrective control system while Figure 6 illustrates an 
open, adaptive control system. In Figure 6, the control system is open because it 
accepts additional input from the "higher authority" which is situated in the 
controlled system's environment. The control system is also adaptive as there is an 
accommodation made for supplying unprogrammed decision rules by the higher 


authority. 


In Figure 6, we can also look at the relative complexity of the first-order control 
(represented by the elements up to the activator) versus the second-order control 
(represented by the "refer to higher authority" loop). The second-order control is 
more complex. First, it can handle a wider variety of states than the first-order 
control and supply decision rules which would otherwise not exist. Furthermore, 

the second-order control can handle a wider range of functions. The second-order 
control system can change all aspects of the lower-order system including the nature 
of the activator. This represents a broader and more complex range of functions 


than those available to the lower-order system. 


We have already mentioned, in the part on general systems concepts, the importance 
to the auditor of identifying whether a system is open or closed, adaptive or 
corrective. It is through examination of the control sub-system that auditors can 


determine what type of system they are actually dealing with. 


8 Summary Statement: Hierarchies of controls can be distinguished 
according to the scale of observation and according to the number and 


complexity of situations or functions that can be handled. 
Information Flows 


Implicit in our control models depicted in Figures 4 through 6 is the transmission of 
information which links the various elements of the control system. Depending 
upon the nature and direction of the flow of control information, further distinctions 


can be made between types of controls. 
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In a feedback situation, control information arises from judgments concerning the 
performance of conversion activities which have already occurred. This information 
is transmitted to the activator which governs the behaviour of the conversion 
processes under review (see Figures 4 and 6 for examples of feedback information 


flows). 


In a feedforward situation, control information is derived before the implementation 
of conversion activities to which it relates. This information is transmitted to an 
activator which adjusts conversion activities that are still to be performed (see 


Figure 5 for an example of a feedforward information flow). 


® Summary Statement: The transmission of control information may be 
characterized as either being feedback or feedforward. 


Summary 


In this part, we have described a model of control using system theory fundamentals. 
We began by describing the purposes, elements and activities thought to be present 

in all effective control systems. We then introduced characteristics that could 

vary between control systems depending upon the strategies adopted to achieve 
control, the complexity of the underlying systemic processes and the nature of the 
system's environment. Figure 7 summarizes in decision-table format a decomposition 
of a control system according to the different variables that have been discussed 

up to this point. This decomposition is meant to classify, according to type, any 


particular manifestation of control. 


The systems techniques, the control model and the decision-table decomposition 
represent tools that internal auditors can use in both describing and evaluating an 
organization's control framework. In Section Three, we will explore how the 
auditor can model organizational activities using the systems and control concepts 


provided. 
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SECTION THREE: CONTROL MODELS FOR AUDITS 


In this section, two organizational control models are presented for use by auditors 
in their examinations. The first part of Section Three relates to the development 
of a descriptive organizational model. Auditors would prepare such a model when 
documenting their understanding of the actual processes subject to audit. The 
discussion begins with organizational modelling and finishes with the presentation 
of how the auditor can build a descriptive model of the processes subject to audit 


incorporating the concepts and techniques provided in Section Two. 


The second part of Section Three provides information relating to the development 
of a prescriptive model. This model contains the standards or criteria through 
which the actual auditee operations will be evaluated. The model is developed 
independently of the descriptive model so that the evaluative criteria are not biased 
by a knowledge of the actual controls. Discussion focuses on the issues related to 
the development and use of the predetermined control model and again employs the 


concepts and techniques provided in Section Two. 
Descriptive Organizational Models 


In Section Two, it was noted that controls are applied only to purposive entities. 

An organization exhibits purpose through its goal-seeking behaviour, and as such, 
needs controls to ensure that the desired state or conditions specified in its goals 
are attained or maintained. Here, the generic control system of Section Two will 


be restated as an organizational model. 


This section begins with a review of the purposes of organizational models from 

the auditor's perspective. It then considers various traditional approaches that 
have been typically used in generating such models and the limitations associated 
with their usage. Finally, the organizational control system model is introduced 
including a discussion of the techniques by which the model can be elaborated. The 
organizational control model uses the same concepts as those used in the generic 
model of Section Two and the paper links the two models on an element-by-element 


basis. 
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Purposes of Organizational Models! 


Models are of particular importance to analysts of all types of organizations. Faced 
with a confusing array of facts, analysts use models to bring order to their compre- 


hension of organizational structure and behaviour. 


Specifically, a descriptive organizational model of the audited entity assists 


auditors in the following ways: 


e communicating within a multi-disciplined group of auditors and between 
the auditors and the auditees (the model provides a common basis for 


understanding the organization); 


e focusing on the organizational elements to be audited (scope of the 
audit); 
a determining and displaying the key operations of the organization under 


audit (raison d'étre of the organization); 


8 determining and displaying the management complement and the 


instruments through which it works (mandate, etc.); 


e focusing on organizational dynamics and cause/effect relationships; 
and 

@ understanding the interaction between the organization and its 
environment. 


In summary, the organizational model serves as a vital tool in the structuring and 


communication of an auditor's approach to an examination. 


15 Audit Services Bureau, Introduction to Operational Audit, Supply and Services 
Canada, 1978, p. 30. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 2, Section 3 - 209 - 


Conventional Approaches to Organizational Modelling 


The organizational model illustrated here uses systems and control concepts. To 
appreciate the value in using a control system approach to modelling organizations, 


this part explores traditional approaches and their limitations. 


At the most fundamental level, organizational activity can be represented by 
numbers, words or pictures. Once the symbol has been chosen, it is then structured 
in accordance with some underlying model. The most traditional one is the financial 
accounting model, based on the premise that each activity in the organization has , 
its monetary equivalent. The monetary unit is used to represent an entity's resources 
and activities. The model is flexible enough to provide both a static and dynamic 
portrayal of an entity's activities through the use of representations such as the 
balance sheet and income statement. Other ways have been used including many 
types of structural representations. The organization chart approach is a typical 
example of a structural model of an organization where different sectors are 
represented according to formal relationships, delegated authorities, responsi- 


bilities and activities. 


The systems approach represents organizational activities according to the input- 
process-output-feedback-environment concepts discussed in Section Two. A significant 
advantage in the use of this approach over traditional models is that the analyst 
must make explicit the assumptions used in developing the model of the system 
under review. Traditional models generally contain fixed assumptions concerning 
the purpose, elements, relationships and boundaries of the entity under review. 

The appropriateness of these fixed assumptions is generally not scrutinized and 
they have improperly become almost the equivalent of universal truths. This lack 
of explicit examination creates the possibility that changing conditions may render 
the assumptions invalid and the model obsolete. In the systems approach, there are 
no fixed assumptions. The analyst is forced to define these aspects. As such, the 
assumptions of the model must be explicitly stated and be readily available for 
review for appropriateness. The need for redefinition of assumptions because of 
changed conditions affecting the entity under review will be more evident using a 


systems approach. 
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Other important advantages can be associated with the use of the systems approach 
over traditional models. First, a systems approach allows specific recognition of 
the organization's environment as an important variable of the model. Second, the 
systems model provides a wider range of options for sub-dividing an entity into 


smaller units for analysis. These advantages are clarified below. 


The financial and organizational structure models tend to be relatively closed to 
considerations about the organization's environment. Generally, in the use of these 
models, the characteristics which define an organization are its internal aspects, 
while external environmental interaction is for the most part considered inconse- 
quential to the definition of the term organization. A systems model typically 
focuses on the organization-environment interface because it is usually assumed 
that the organization is an open system which cannot be defined exclusively by its 
internal characteristics. Specific consideration of the organization's interface with 
its environment is becoming increasingly important given the growing complexity 


and dynamic character of these relationships. 


Conventional models tend to be structured on the basis of a limited number of 
criteria. For example, organization chart models are based on a sub-division of 
formal authorities, relationships and responsibilities. A systems model is based on 
common purpose and includes all parts and all interactions involved to bring this 
purpose about. This wide representation of types of relationships in the model 
provides a greater richness in the number of sub-systems that can be defined for 
more in-depth analysis. When studying aspects of an organization, the breadth of 
the systems approach is clearly advantageous. For example, conventional models 
which sub-divide the organization according to its structure cannot easily deal with 
subjects, such as information flows, that cross formal organizational divisions. The 
study of information flows or other matters that cross the basis for sub-dividing the 
whole is performed awkwardly and often in a piecemeal, specialized manner. A 
systems approach, being more flexible in the alternative ways in which sub-systems 
are defined, can readily structure an information flow sub-system for detailed 


analysis. 


16 Schoderbek, Peter P., Kefalas, Asterios G. and Schoderbek, Charles G., 


Management Systems Conceptual Considerations, Dallas: Business 
Publications, Inc. 1975, pp. 125-126. 
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With these advantages in mind, the next part identifies a major concern relating to 
the use of a systems approach in modelling organizations and some of the techniques 


that can be employed to help ensure that a credible model is developed. 


Dealing with Systems Modelling Concerns 


In general, the analyst should recognize that one problem with the use of any model 
is the difficulty of representing in simpler form the enormously complex data 
relating to an organization. A systems model is not unique in this respect and some 
difficulty will likely be experienced in fitting information concerning the 
organization into the input-process-output-feedback-environment scheme. Three 


activities may help the analyst achieve a credible systems model. 


First, the analyst must clearly define in an organizational context precisely what 
data are to be accumulated under each system element. In the next part, the auditor 
is given some help in this regard. The control concepts discussed in Section Two 

are redefined to reflect their nature in an organizational setting. These definitions 
will provide one example of the way in which auditors can initially identify a frame- 


work for the accumulation and classification of organizational information. 


A second activity in developing a credible systems model is to develop an under- 
standing by using a series of models which start at a high level of abstraction and 
move progressively to greater and greater levels of detail for particular aspects of 
the more broadly conceived system. The use of this technique is illustrated in 


Figure 8. 


This example considers the case of an auditor wishing to do an audit of a division of 
a department. Asa start to developing an understanding of the division, the auditor 
may construct a simple overview model of the entire department. The elements at 
this level are broadly defined and give a picture of overall departmental mission, 
objectives, strategies and major programs. At the next level of resolution, the 
auditor may model the one branch that contains the division being audited. Greater 
detail is brought to the original descriptions concerning mission, objectives, 


strategies, etc. A third model, that of the division itself, would provide even 
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greater levels of detail. This technique of progressive elaboration helps cope with 
the vast amount of information that must be dealt with during an audit. The auditor 
retains an appreciation of the interrelationship of the entity under review with 
other organizational elements, but avoids overemphasis of modelling aspects of the 


system that are relevant only to a limited degree. 


Modelling through progressive elaboration is used to illustrate how each of the 
generic control elements of the organization model can be brought into focus. 
Subsequent to the definition of the elements of the organization control model in 

the next part, an elaboration is provided as to how each element of the most abstract 


model can be defined in a more detailed way. 


A third activity which helps ensure that the auditor's representation of an organi- 
zation is appropriate involves confirming the accuracy of the model with the auditee. 
This technique should be familiar to all auditors and will not be given further elabo- 


ration. 


An Organizational Control System Model 


As noted, the development of an organizational control model first depends on 

proper definition of the elements of the system involved. The control concepts 
discussed in Section Two are redefined here to reflect their nature in an organiza- 
tional setting. The definitions used for each of the control elements have been 
adapted from the Audit Services Bureau's organizational model. It is stressed that, 
though its definitions have survived the test of time, they represent only one example 
of the many ways in which a generic control model can be expressed in organizational 


terms. 


The definition of organizational control terms are reflected in Table 1. An initial 
layout of the control system is also shown in Figure 9. Once this structure is 


presented, subsequent parts go on to elaborate upon these initial terms. 
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Table | 


General versus Organizational Control System Elements 


General Control Organizational Control System Element!” 
Systern Element (adapted from Audit Services Bureau model) 
(from Section One) 
Environment Environment 
- is external to the boundaries - all conditions, circumstances, etc., 
of the system structure; surrounding and affecting the develop- 
ment of an organization; 
- relevant in that it influences - the essential conditions may be 
and can be influenced by the categorized along the following 
system's behaviour; divisions: 
i) environmental needs 
ii) environmental resources 
iii) environmental constraints 
iv) environmental influences; 
- not controllable by the - environmental needs may be a product 
system. or service that is useful to society 


but is currently unavailable; 


- environmental resources include 
money, materials, machines, people 
and ideas; 


- environmental constraints include 
laws and regulations; and’ 


- environmental influences include 
political, social, economic, 
technological, moral and ethical 
influences. 


17. + Audit Services Bureau, op. cit., pp. 17-27. 
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Table 1 (cont'd) 


General versus Organizational Control System Elements 


General Control Organizational Control System Element!” 

System Element 

Higher Authority Management 

- responsible for the design, - this model assumes that regardless of 
implementation and control the actual number of managerial 
of the processes of the levels in an organization, they can be 
underlying system; fit into two general categories: 

i) general management 
ii) operational management; 

- in an open system, information - general management is concerned 
from the environment is with the perception of the environment 
accepted for consideration by and the definition of the appropriate 
the higher authority. mission, objectives and strategic 


plans of the organization; 


- this level issues policies, directives 
and guidelines to operational manage- 
ment to provide a framework for 
action; 


- essentially it represents a second- 
order higher authority over the actual 
organizational operations; 


~ operational management translates 


the policies, directives and guidelines 
into operational plans, procedures, 
techniques and performance standards 
which govern the deployment of 
resources, the method of operations 
and their control; 


- essentially this level operates as a 
first-order higher authority over the 
actual organizational operations. 
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Table | (cont'd) 


General versus Organizational Control System Elements 


General Control Organizational Control System Element!’ 


System Element 


Inputs Inputs 


- they are the start-up force 
that provides the system with 
its operating necessities 
be they material, energy, 
demands for products and 
services, humans, or simply 
information; 


- can be outputs of other systems 
or a reintroduction of a portion 
of the output of the same 
system. 


Process 


- that which transforms 
input to output. 


goods, data or resources which come 
from outside the boundaries of the 
organization; 


they initiate or are used in a work 
process; 


resources may include finances (cash, 
credit), equipment (production, office), 
materials (raw materials, parts, 
components), staff, technology 

(patents, copyrights, methodology, 
processes, techniques, ideas), intangibles 
(goodwill, location) and physical 
facilities (land, buildings). 


Transformation Processes 


all activities necessary to convert 
resources (input) to finished marketable 
products or services (output); 


two systems: production and support; 


production system: involved directly 


in the preparation of the organization's 
products or services; 


support system: involved in acquisition, 
maintenance, and disposal of the 
organization's internal resources. 
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Table | (cont'd) 


General versus Organizational Control System Elements 


General Control Organizational Control System Elements!” 

System Elements 

Outputs Outputs 

- products, services, information - products of operations. 
or energy resultant from 
systemic processes. 

Information (including Feedback) Information (including Control Data) 

- information is the force which - means of linking various elements of 
links each element of the the organizational model together; 
system together; 

- feedback is found in all - organizational information is captured 
purposive systems and is used within two classes of systems: 
to ensure that the desired operations information and management 
state of the system is information; 


maintained or attained; 


- feedback represents the ~ operations information systems: 
control function of a system; communicates operations data to 


operating personnel to identify what 
is happening at operations levels; 


- as a sub-system, the control ~ also communicates control data which 
function includes the following represents to operating personnel and 
elements: control object, detector, management the degree to which 
reference point, comparator, operations conform to expectations; 
activator and higher authority 
(previously mentioned). - management information systems: 


supplies information to the decision- 
making processes which are involved 
in the interaction between the 
organization's operations and manage- 
ment and between the organization 
and its environment. 
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ORGANIZATION CONTROL MODEL!® 
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Figure 9 


18 Adapted from Audit Services Bureau, A Structured Methodology for the 


Conduct of Comprehensive Auditing, Supply and Services Canada, 1982, 
Annex A Diagram 8. 
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Initial Adjustments to the Organizational Model 


The organizational model shown in Figure 9 represents a useful variant of a systems 
model and it is presently used by a number of federal government internal audit 
groups. Being a type of systems model, it allows the integration of the system and 
control concepts raised in Section Two. Figure 10 represents a modification of 

Figure 9 which visibly incorporates the system and control concepts of Section Two 
into the overall framework provided by Figure 9. While some of the detail of Figure 9 
has been omitted from the revised model to highlight the modifications made, it is 
assumed that the excluded details will be incorporated into the actual model developed 


by the auditor. 
The following modifications have been made in Figure 10: 


& The general management and operational management elements are 
clearly distinguished as controllable processes with identifiable inputs, 
outputs and feedback loops. As such, the management processes are 
identified as special transformation processes from which management 
outputs are derived. Because of the importance of management outputs 
to the design, operation and control of the organizational processes, 
specific identification of the processes which generate these outputs is 
important. An adequate representation of management processes provides 
the auditor with a basis for evaluating whether desirable management 
outputs can be produced. The management processes are sub-systems of 
the larger organizational system and their output generally becomes 


input for the next internal sub-system. 


8 There is specific recognition of a higher authority in the environment of 
the entity under review. Essentially, this elaboration differentiates 
between general environmental influences and influences from a higher 
authority. In most situations, persons performing strategic general 
management functions for the entity under review take instruction 
from, and are accountable to, a higher authority. This relationship is 
considered sufficiently significant to warrant separate recognition. The 
higher authority properly belongs in the environment of the entity under 


review, in that it is not subject to control by that entity. 
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AN ORGANIZATIONAL CONTROL MODEL - MODIFIED VERSION ‘ 
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Depending upon the observer's point of view, the general and operational 
management elements can also be viewed as higher authorities. When 
the operational management element is viewed as the operating system 
(e.g. is the focus of the audit), the general management element becomes 
a higher authority situated in the environment of the operating system 
under review. Likewise the operational management element can be 


viewed as a higher authority over the transformation processes. 


® The control data loop from Figure 9 has been given greater detail, in 
accordance with the control system model in Section Two. The auditor's 
primary perspective for evaluation is through the control framework. 
Consequently, elaborations of the elements and functions of the control 
sub-systems provide auditors with a more concrete basis for their review. 
For convenience in presentation, all of the control loops shown in 
Figure 10 are represented as the same type of sub-system. They are 
located at the output position of the process they are controlling. The 
controls are all open in that they can accept uncontrollable input from 
a higher authority. Recalling from Section One the various characteriza- 
tions that controls can exhibit, it will be recognized that Figure 10 
could be adjusted to include varying types of control sub-systems (e.g. 


closed with feedforward loops, located over input or conversion processes). 


Secondary Elaborations on the Organizational Model 


According to the discussion provided earlier, the initial definition of organizational 
terms requires elaboration if a clear understanding of the processes under review is 
to be achieved. Each of the primary elements of the model shown in Figure 10 can 
be looked at individually so as to expand the analyst's awareness of the processes at 
work. Figure 11 represents an expanded view of the general management element 


included in the original model. 


In Figure 11, the general management element of the overall model is shown to 
include the processes of planning, implementing and controlling. A new model, in 
effect, has been created to describe one element of the organization model. Such 


elaborations should be performed for all other elements of Figure 10. To illustrate 
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this point, an expanded view of modelling the management elements and control 
loops of the organization model follows. Reference will also be made to other 


possible elaborations. 
i) Detailed Models of the Managerial Elements 


Both the general management and the operational management elements of the 
organizational model represent processes which produce their outputs on the basis 
of managerial action. These elements act as an important link between the organi- 
zational model and the extensive literature available on the modelling of managerial 
activities. Essentially, the organizational model establishes the context surrounding 
managerial activities. The managerial model increases the detail of information 


gathered and classified for one element of the model. 


Even the most preliminary survey of management literature will reveal the many 
alternative methods available for modelling management activities. Table 2 
summarizes some of the various proposals made. While no real consensus exists, 
four functions (planning, organizing, directing and controlling) have gained wide- 
spread recognition. Our conclusion is that the use of these categories of managerial 
activities will be beneficial to the auditor when observing and categorizing infor- 
mation relating to the nature of managerial work. This conclusion is reflected in 
the detailed model of the general management element in Figure 11. The activities 
of the manager are described by the three functions planning, implementing and 
controlling where the term implementing represents the classification of the tasks 


of organizing and directing under a single heading. 


In addition to identifying broad categories for classifying management activities, 
the auditor must also determine an adequate description of activities included 
within each managerial category. Management writers have not agreed upon a 
definition for any of the managerial processes with any great degree of precision 


and again the auditor is left with a decision of choosing among available alternatives. 


For a more detailed discussion of the relationship of management and control refer 


to Chapter 3 of this Volume, entitled "Management Control: Concepts and Practices". 
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Lists of Management Functions 


Dale 


Planning 
Organizing 
Staffing 
Directing 
Controlling 
Innovating 
Representing 


Greenwood 


Planning 
Decision-making 
Organizing 

Staffing 

Directing and Leading 
Controlling 


Gross 


Decision-making 
Communicating 
Planning 
Activating 
Evaluating 


Johnson, Kast, and Rosenzweig 


Planning 
Organizing 
Controlling 
Communication 


~ 290% 


Table 2 
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Koontz and O'Donnell 


Planning 
Organizing 
Staffing 
Directing 
Controlling 


Longenecker 


Planning 

Organizing 

Directing and Motivating 
Controlling 


Massie 


Decision-making 
Organizing 
Staffing 
Planning 
Controlling 
Communicating 
Directing 


Mintzberg 


Interpersonal Role 
Informational Role 
Decisional Role 


Newman, Summer, and Warren 


Organizing 
Planning 
Leading 
Controlling 


Voich and Wren 


Planning 
Organizing 
Controlling 
Administering 


Miner, John B., The Management Process - Theory, Research, and Practice, 
New York: The MacMillan Company 1973, p. 48. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 2, Section 3 - 225 - 


ii)  Elaborations of Control Loops 


The control loops attached to each process in Figure 10 represent the other 
important sub-systems within the overall organizational control system model. As 
will be shown subsequently, it is through the evaluation of these control sub-systems 
that the auditor determines the adequacy of the organizational processes and 
managerial concern for results. As such, it is evident that the auditor must obtain 


an adequate description of the control sub-system to provide a basis for evaluation. 


The elements of the organizational contro! sub-system are consistent with those of 
the generic control system depicted in Figure 4. The following points represent a 
guide to modelling each of these control elements in an organizational context. 
These points are meant to be illustrative and do not represent a complete discussion 


of modelling control sub-systems.-7 


Object of Control: As noted in Section Two, the control object is the aspect of the 
system's behaviour chosen for monitoring and control. The choice of the control 

object is a key concern of the manager as it determines the behaviour of the whole 
control sub-system. The control object can be found at the input, process or output 


locations of the entity under control. 


The auditor's primary interest in gathering information on the control object is to 


answer the following questions: 


a which aspects of the system's behaviour are monitored and controlled? which 
aspects are not? (consider the dimensions of quantity, quality, timing and 


cost). 


6 which control objective is meant to be served through the observation and 


control of the identified control objects? 


20  Edds, John A., Management Auditing Concepts and Practice, Dubuque, Iowa 
and Toronto, Ontario: Kendall/Hunt Publishing Company 1980, pp. 172-173. 
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Detector: The detector sub-system scans the control object and feeds this 


information to the other elements of the control sub-system. 


The auditor would gather the following information on the detector: 


attribute being measured; 

frequency with which information is gathered; 
capacity of the detector to absorb information; 
efficiency of the detection processes; 


precision and accuracy of information detected; and 


cost of detection. 


Literature relating to management information systems provides an additional basis 


for gathering more specialized information relating to the detection function. 


Reference Points: In an organization, these represent the benchmarks established 
for acceptable system outputs and behaviour. The auditor would look for and 
describe standards relating to the aspects of the system's outputs and behaviour 


being monitored (e.g. quantity, quality, timeliness, cost). 


Comparator: The function of the comparator is to compare actual performance 


against an identified standard. 
The auditor would be interested in: 
& the frequency of comparisons of actual against desired; and 


& the sensitivity of the comparator - the auditor would note how much 
the control object is allowed to vary before a signal is generated 


identifying a deviation in the system's behaviour. 


Activator: The activator is a decision-making mechanism (or decision-maker if a 
person) which acts in accordance with predetermined rules to correct deviant system 


behaviour. When no decision rule exists, the activator refers the deviation toa 
higher authority. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 2, Section 3 - 227 - 


The auditor would gather information on: 


& the nature and frequency of deviations requiring response and how they 


are handled; 
the effects of the activator's response; 


e the nature and frequency of deviations referred to a higher authority 


and the effects of such referrals; and 


e the nature of the activator's decision-making processes where some 


discretion is allowed. 


Varying Design Considerations: In Section Two, we noted that certain distinctions 
can be made in describing the detailed characteristics associated with various 
controls. Auditors, by including a description of whether controls are preemptive, 
preventative, or detective, open or closed, adaptive or corrective, feedforward or 
feedback, will establish a better basis for their subsequent evaluation of the 
adequacy of the controls. Auditors should also include in their models an indication 
of whether the nature of the operating system and its environment are consistent 
with the type of control system in place (Section Two provided some guidance in 


this regard). 
iii) Elaborating Other Elements of the Organization Model 


Just as the management elements and control loops were subject to elaboration, so 
also can all other elements of the overall organization model be further described 
in detail. While it is beyond the scope of this paper to suggest detailed models for 


each of the other major elements, a few comments can be made. 


With respect to the organization-environment interface, the auditor is most 
interested in identifying key environmental influences and the nature of the scanning 
system employed by the organization to maintain and regulate information 


concerning the environment. 
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Information flows between the elements of the organization model would also be of 
concern to the auditor. The information flows can be modelled as unique sub- 


systems containing inputs, processes, outputs and control loops. 


Finally, the transformation processes can be described in detail by the auditor. 

The auditor attempts to depict operational processes to the extent that they are 
predictable or prescribed and can be related to outputs at least in a general manner. 
This depiction is normally accomplished through the use of narrative description, 


flowcharting techniques or decision tables. 
Summary 


In this first part of Section Three, attention has been centred on developing an 
organizational model which can be used by the auditor when developing an under- 
standing of the entity under review. To this point, the auditor has been involved in 
describing the entity without concern for judging the adequacy of the processes 


reviewed. 


In the development of descriptive models, it has been suggested that the auditor 
clearly define the elements under review, use a series of progressively more 
elaborate models and verify the accuracy of completed models with the auditee. 
Because the auditor, in preparing a descriptive model, is concerned with "what is 
going on" in the entity under review and not "what should be going on", there is 
general encouragement to draw freely from management literature as an aid to 


sorting out complex data. 


The balance of Section Three is devoted to the development of a prescriptive or 
normative model which provides the auditor with the basis for determining the 
adequacy of the auditee's control framework. The normative model contains 
standards of expected performance which, when applied against the auditor's 
descriptive model, provide an initial basis for concluding on the adequacy of the 
auditee's operations. In the audit environment, this model is normally referred to 


as the predetermined control model. 
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Prescriptive Organizational Models 


Prescriptive models contain the value system or standards that an auditor uses to 
evaluate the adequacy of the entity under review. The development of the 
prescriptive model represents a critical component of the audit assignment process 
since the ultimate quality of the audit report and its usefulness to management will 
depend on the soundness and relevance of the standards or criteria the model 
employs. One type of prescriptive model used by auditors is called a predetermined 
control model. 


Predetermined Control Model - Outline of its Contents 
As envisaged in this chapter, the predetermined control model should identify those 


controls considered essential to the operations of the audited entity. For each 


control within the model, there should be: 


& an explicit statement of the objective served by the control; 
@ standards relating to what constitutes proper control design; 
@ indicators which will help establish whether the control is operating 


effectively (i.e. criteria for judging adequacy of performance). 


With this information, the auditor has a satisfactory basis for evaluating the 
operations of the auditee. Comparison of actual processes and controls of the 
auditee operations with the standards established within the predetermined control 
framework should allow the auditor to conclude whether all essential controls exist, 


are properly designed and operating effectively. 


Responsibility for Preparing the Predetermined Control Model 


Criteria used to judge the adequacy of controls have not been codified in the form 
of generally accepted principles for many areas subject to audit by internal 
auditors. To a large extent, the criteria embedded in the predetermined control 


model must be first chosen on a judgmental basis by the internal auditor. 
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While auditor judgment is necessary in the determination of criteria, internal 
auditors should not unilaterally establish the criteria to be used on any given assign- 
ment. Instead, auditors should seek input from the intended recipients of the audit 
report and from auditees before determining the criteria to be used. In addition to 
this input, auditors should also attempt to ensure that all criteria used are supported 
through reference to an authoritative source (e.g. legislation, central agency and 
departmental policies, applicable generally accepted practices, etc.) where appro- 


priate and feasible. 


Obtaining input from the intended recipient of the audit report is particularly useful 
in ensuring that the final audit product is relevant to the persons served by the 
audit process. In the federal government, the internal auditor performs audits 
primarily for a deputy minister or a department's senior officials. Secondary users 
of internal audit reports would include departmental managers subordinate to those 
noted above and external audit or central agency review groups who may rely on 
the work performed by the internal auditor. Input from these users can help the 
auditor in decisions relating to what areas should be emphasized in the auditor's 


examination and what criteria may apply for evaluating performance. 


Obtaining input from auditees is also of use in the establishment of audit criteria 
because of their intimate understanding of the operations subject to review. Because 
of the vested interest of the auditee in the outcome of the auditor's assessment, 

they are more useful when employed as a challenge process to the soundness of the 


criteria developed by the auditor than as an original source of information. 


Guidelines for Developing the Predetermined Control Model 


Two specific problems are involved in the development of a predetermined control 


model: 
e How does an auditor determine whether all essential controls have been 
included in the model? 
% What types of criteria can be used for assessing the adequacy of control 


design and effectiveness? 
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This part will explore possible ways the auditor may attempt to find answers to 


these questions. 


i) Identifying Essential Controls 


One of the most difficult tasks facing the auditor is determination of what controls 
should be subject to evaluation. Because of the complexity of the operations the 


auditor is often facing, the task should be approached as systematically as possible. 


In reviewing any organizational activity, the auditor can initially view it as a system 
which is attempting to achieve certain results through a given process. The auditor 
is interested in assessing the adequacy of the controls designed to ensure that 
intended results are achieved and any process procedures required to produce these 
results are being properly performed. At a coarse level of resolution, then, auditors 


are concerned with the adequacy of controls over results and controls over process. 


The auditor can further refine this initial decomposition of the auditee's operations. 
Using the systems concepts noted in this chapter, processes can be looked at as 
containing inputs, conversion activities and outputs. In the federal government, 
statements about results can typically be reconciled to the three dimensions of 
economy, efficiency and effectiveness. These further divisions of the auditee's 
operations help auditors focus in on the types of controls that relate to more specific 


aspects of the system under review. 


For each sub-division of the auditee's processes and results noted above, the auditor 
should determine what specific objectives management of that operation is trying 
to achieve. Given an understanding of these objectives, the auditor should then 
assess what could go wrong in the pursuit of these management objectives. In other 


words, where are controls needed? 


Table 3 identifies in broad terms typical control objectives that one could likely 
derive from using the general questioning process noted above. The intention of 


this illustration is to provide a general classification scheme for various control 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 2, Section 3 - 232 - 


objectives and where they are most likely to apply in the system under review. 
Auditing texts often enumerate more specific control objectives which one should 


be able to integrate within this general scheme. 
® Identifying Auditee Processes 


Implicit in the above discussion is the assumption that the auditor is reviewing 
a system which contains only one process. Earlier in this chapter, the develop- 
ment of a descriptive model of the auditee involved three processes: a 
general management process, an operational management process and the 
operational process itself. It is contended here that these three processes 

will typically be found in most entities subject to audit. Consequently, the 
predetermined control model should typically be multi-dimensional, identifying 
control objectives for each of the several processes normally contained within 


a single audit entity. 


It should also be recognized that predetermined control models are developed 
not only to audit existing systems and processes but can be used as a template 


for pre-implementation audits of systems under development. 
® Materiality and Evaluation of Controls 


The procedures to this point will provide the auditor with a method for 
identifying the range of control objectives that can be subjected to the 
auditor's examination. The auditor, however, does not evaluate all auditee 
controls but must choose for examination those controls which relate to items 


which are of "material" concern to the recipient of the audit report. 


’ 
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Table 3 


Typical Control Objectives 


A) Process Control Objectives 


Input* Control Conversion 
Objectives (Transformation Process)* 
Control Objectives 


Controls to ensure: Controls to ensure: 

- Selection and - Prevention or detection of 
maintenance of accidental errors in 
suitable infra- conversion of input to 
structure, resources output; 


and raw materials; 
- Prevention or detection of 


- Appropriate handling fraudulent activities 
of demands for during the conversion of 
Service. input to output; 


- Security of conversion 
process to ensure 
continuous operation; 


- Compliance with authorities 
and prescribed policies and 


procedures. 
B) Results Control Objectives 
Economy Control Efficiency Control 
Objectives Objectives 
Controls to ensure Controls to ensure 
that resource that output achieved 
acquisition and for input supplied is 
maintenance are optimal. 


done economically. 


* Refer to Table | (page 214-217) for definitions. 


Output* Control 
Objectives 


Controls to ensure: 


- Completeness of 
output; 


- Accuracy or 
freedom from 
error in outputs; 


- Timeliness and 
appropriate 
distribution 
of output; 


- Compliance with 
specifications 
from higher 
authority. 


Effectiveness 
Control Objectives 


Controls to ensure 
that entity 

output is according 
to plans and has 
the desired effects. 
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In internal auditing "... an item would be considered material if an error in it (or its 
complete omission) would cause prudent, intelligent information users to change 
decisions that they might otherwise make on the basis of information provided by 


and about the auditee".2! 


The practical problem for internal auditors is to determine whether errors would 
significantly influence the decision-making process of users should they be disclosed. 
Failure to establish a proper materiality limit will result in the gathering of audit 
evidence and the reporting of audit results at an inappropriate level of detail. A 
threshold limit which is set at too high a level of detail will result in audit report 
comments which are too general to be of significant use to the reader of the audit 
report. Threshold limits which are too low can result in over-auditing and the 


cluttering up of audit reports with information at an overly precise level of detail. 


Internal auditors must use their professional judgrnent when establishing the level 
of detail covered by their review. Table 4 lists a number of factors the auditor 

may wish to consider when determining which items in the sytem under review 
constitute material ones and consequently require inclusion of their related controls 


in the predetermined control model. 


It should be recognized that many controls are useful though not deemed material 
by the auditor. The auditor's viewpoint on the significance of controls incorporates 
the deputy minister's perspective. Clearly, lower-level managers establish controls 
which are necessary for the proper performance of their function. Care must be 
taken, therefore, in avoiding the assumption that when a control is not deemed 
material by the auditor, it serves no useful purpose and should be discarded. There 
will often be many useful controls which may not be considered significant from an 


audit standpoint. 


24: ‘Eds onvcitiip. 144, 
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Table 4 
Materiality Guidelines 
The following represent factors the auditor should consider in determining whether 


items under review are material. The auditor, with appropriate input from users, 


should review items in terms of whether they involve: 


e known or probable management issues or concerns; 

e areas of particular on-going concern identified in previous audits; 

® questions of non-compliance with financial regulations; 

® fraud or other irregularities; 

e potential areas of uneconomical or inefficient operations; 

® uncertainty concerning the entity's knowledge of the effectiveness of 


its programs and deficiencies in the entity's procedures to evaluate 


effectiveness; 


a a program or activity of particular interest because of its nature or 


relative size, and its importance or impact; 


€ significant new or expanded programs or activities; 


% unusual program management characteristics, such as restrictions or 


freedoms in carrying out functions; and 


e financial, human and physical resources of particular interest because 


of their nature and importance. 


It is likely that where an item involves at least one of these factors, it will be 
"material" to the user of the audit report. The predetermined control model should 


include all controls relating to material items. 
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ii) Criteria for Control Design and Effectiveness 


The predetermined control model should contain criteria relating to what constitutes 
the proper design for controls established to achieve the control objectives 

selected by the auditor for examination. In addition, indicators which would identify 
effective operation of the controls under review should be included in the auditor's 
criteria. This information forms the basis of the auditor's evaluation of the 


adequacy of controls (the method for this evaluation is discussed in Section Four). 


In terms of control design criteria, the auditor should first determine what type of 
control would be appropriate given the control requirements of the underlying system. 
In Section Two, alternative design strategies for control systems were discussed 
(these are summarized in points 2 and 3 of Figure 7, page 206). The auditor must 
choose from the options available those control designs that are appropriate to the 
system under review and reflect the need for such control designs by establishing 


them as criteria within the predetermined control model. 


The criteria for control design should also provide the basis for reviewing whether 
all necessary elements in a control system exist (refer to Figure 6, page 204) and are 
operating properly (e.g. performing adequately the functions normally associated 


with each element or as prescribed by laid-down policies and procedures). 


Table 5 illustrates an approach that could be taken in the development of criteria 


relating to contro! design. 


The final aspect of the predetermined control model to be developed are criteria 
which can be used to determine the control's effectiveness. Criteria relating to 


effectiveness are: 


a the extent to which the control objectives established by management 


have been achieved; 


6 the appropriateness of the established control objectives in the first 


place, given the control needs of the system under review. 
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The actual criteria used to measure effectiveness are often statements which 
relate to the types of conditions one would expect to find in the system if it were 
subject to proper control. With such standards for performance, the auditor would 
gather evidence which provides positive proof of a control's effectiveness. 
Conversely, criteria may be statements relating to the types of conditions that 
indicate the system is out of control. Absence of evidence relating to control 


problems also provides indirect proof of control effectiveness. 
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Table 5 


Development of Criteria Relating to Control Design 


A. Overall Control Design Considerations 


Given the nature of the specific control objective under review, provide 


criteria which indicate that the design of the control system should be either: 


_@ preemptive, preventative or detective; 


@ open or closed to input from the environment of the system under review; 


identify extent; 


a adaptive or corrective; 


e feedback or feedforward. 


B. | Completeness of Control and Proper Performance by All Control Elements 


2 identify the existence of the elements of a control system: control 


object, detector, reference point, comparator and activator 


* provide criteria which can be used to assess the adequacy of the function 


performed by each element; consider: 


i) control object 


~ identify the aspects of the system's processes and results that 
should be monitored (consider the dimensions of quantity, quality, 
timing and cost or other attribute for which there is a predefined 


requirement); 
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Table 5 (Cont'd) 
Development of Criteria Relating to Control Design 
ii) detector 
- identify the criteria which indicate the manner by which information 
on what is actually happening in the system under control should 
be gathered; 
iii) reference point 
- identify the standards or benchmarks that should be used in the 
control system to determine acceptable system processes and 
results; 
iv) comparator 
- identify the criteria which indicate the manner in which actual 
performance in the system under contro! should be compared 
against desired performance; 
v) activator 
- identify the criteria which indicate the manner in which deviations 
of actual system performance from desired performance are to be 
acted upon. 
Greater detail on the functions of each control element were provided earlier. 
This information may be used to refine the criteria used to evaluate control design. 


Wherever policies and procedures prescribe the manner in which the above elements 


function, these requirements should be built into the control design criterion. 
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Issues Related to Developing the Predetermined Control! Model 


In private-sector, external financial audit practice, the problem of identifying 


where controls should exist and why, has been largely resolved. The following reasons 


have contributed to the existence of a reasonably uniform approach to defining 


control points and objectives relating to a financial system: 


the processes which lead to financial reports are largely prescribed 
through Generally Accepted Accounting Principles (GAAP) and other 
conventions; as such, the locations of process controls are reasonably 


clear and follow the prescribed rules for proper accounting practices; 


the output of the financial reporting system, the financial statements, 
are the output measures of the private sector organization's overall 
objective of surviving and earning a profit; the adequacy of management's 
concern for controlling results is directly observable through review of 
the degree to which financial results are controlled. As in the case of 
process controls, the determination of which results should be controlled 


is therefore also reasonably clear in the private sector; 


the scope of financial information systems is limited in terms of the 
types of activities and their complexity; consequently, an understanding 
of financial processes and their relationship to results can be reasonably 
complete and lends itself to more accurate determination of required 


control points; and 


it is somewhat easier in an accounting context to determine the risks or 
effects of missing controls; controls are meant to protect the financial 
interests of the firm and the effects of control weaknesses can be usually 
quantified in financial terms; being able to establish the effects from 
control deficiencies reduces the likelihood of disagreement as to the 


need for, and location of, controls. 
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While the financial control model can be largely incorporated into corresponding 
aspects of internal auditing, there is a lack of general agreement by public sector 
auditors on the process and results controls required for the broader aspects of the 
auditee's operations subject to the internal auditor's examination. Consequently, 
the degree of confidence relating to the need for, position of, and objectives of 
control is less tangible. Consider, for example, some of the following questions 


regarding the placement of controls over operational or managerial activities: 


® What performance indicators should be used to determine that the 


organization is moving towards achievement of its goals? 


For profit-based, private sector organizations, we noted that financial results were 
tangible indicators of the firm's ability to meet its goals. Consequently, control 
should be established over financial results. In the public sector, progress towards 
the achievement of social goals is normally more pertinent and is also more difficult 
to measure. Problems exist in determining what results indicate satisfactory 
performance and exactly how these results should be measured. These problems 
make the predetermination of the necessary controls over results more difficult for 


the auditor. 


® What agreement is there as to the activities that should be performed 
and controlled to ensure that the desired organizational results are 


achieved? 


Unlike accounting practice, there does not yet exist a comprehensive body of 
managerial principles that are generally accepted as the means by which desired 
outputs should be achieved. In addition, there are no management theories which 
provide absolute truths regarding the manner in which managers should behave. As 
such, the activities in the management process which should be performed and 
controlled cannot be fully established by reference to outside conventions or universal 


management principles. 


It has also been noted that the risks or effects associated with missing accounting 
controls can be reasonably well defined. In the broader operational and managerial 


context, the relationship between particular processes and results is not as clear. 
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As such, the possibility of disagreement concerning the need for a particular practice 
or control may be greater given the fact that risks can be derived less precisely or 


quantitatively. 


Despite the general reservations noted above, there exist a number of managerial 
and operational conventions which provide partial frameworks for establishing the 
types of processes and controls considered necessary to achieve desired results. In 
the federal government, for example, Treasury Board and departmental policies 
and directives establish norms for the nature and types of managerial and operational 
processes and controls that are expected to exist for at least some of the activities 
(e.g. common services like financial, personnel, etc.). In these cases, the auditor 
has a more concrete basis for determining what activities are necessary to achieve 
desired results and where process controls should be established. Outside of these 
areas, where managerial and operational convention do not exist, the auditor is 
again faced with difficult problems regarding the predetermination of necessary 


controls. 
Summary 


We have now reached a point where the auditor has some tools and techniques for 
the development of a descriptive organizational control mode! and a separate 
prescriptive predetermined control framework. These organizational models 

were developed on the basis of the systems modelling and control concepts of 
Section Two. As a guiding principle, the ultimate justification for any model must 
rest on its usefulness in aiding auditors in the performance of their duties. In 


Section Four, use of the control models by the auditor will be explored. 
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SECTION FOUR: THE AUDITOR'S USE OF CONTROL MODELS 


Evaluating the Adequacy of Control Systems 


The auditor evaluates the audited entity's control framework to arrive at 


conclusions in relation to whether: 


® the results of operations are adequately controlled such that actual 


results compare favourably with desired results; and 


® the operational process is satisfactory such that it remains capable of 


achieving desired results over time. 


Using the modelling techniques provided in this document, the auditor will be in 
possession of a descriptive model of the structure and operation of the entity under 
review and a prescriptive model of the controls that should be operating. This 
modelling would essentially be completed by the end of the review phase of the 


audit. 


In the evaluation phase, the auditor evaluates the adequacy of the control 
framework by matching existing controls with the predetermined control mode! and 


answering the following questions: 


Do all essential controls exist? 
Are all essential controls properly designed? 


Are the controls operating as designed? 


Are the controls effective? 


The remainder of this part will discuss preliminary considerations which will aid 


the auditor in dealing with these questions. 
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Do All Essential Controls Exist? 


In the auditor's predetermined control model, all necessary control points and their 
objectives are identified. These controls and their purposes should have been 
legitimized through reference to supporting authoritative sources (i.e. the controls 
have been mutually agreed to by managers or are supported by prescribed policies 
and directives). As an initial test of the adequacy of controls, the auditor would 
match the predeterrhined control model against the descriptive model of the 


organizational control system. 
The result of this matching will give the auditor a preliminary indication of: 
® missing controls at points where essential controls should exist; 


@ the existence of controls at points where no controls are considered 


essential; and 
® the existence of controls at points where essential controls should exist. 


Where an essential control is found missing, the auditor would determine whether 
any adequate compensating control exists. Where a compensating control is found 
the auditor would proceed to test the adequacy of its design, operation and 
effectiveness. Where there are no compensating controls, the auditor would perform 
tests to determine the cause and effects of the noted deficiency. A missing results 
control indicates that management has not been providing sufficient regard to 
ensure that actual results compare favourably to desired results. The auditor may 
be able to substantiate this indication through a direct review of actual results in 
relation to desired results. A missing process control indicates that the capability 
of the process to continue to produce desired results may be in jeopardy. Again, the 
auditor may wish to substantiate this claim by relating deficient process controls 


as a cause of undesirable results. 


Where a control is found at a point where no control is considered essential, the 


auditor would likely test it from a cost/benefit standpoint. 
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Where a control is found at points where controls are expected to be found the 
auditor would proceed with tests relating to adequacy of its design, operation and 


effectiveness. 


Are All Essential Controls Properly Designed? 


In Section Three, guidelines were provided for developing criteria that can be used 
to evaluate the adequacy of control design (refer to Table 5). Criteria related to 
two aspects of design. First, evaluation focuses on whether the overall design of 
the control is appropriate given the nature of the control objective being sought.. 
Second, the auditor must assess whether all elements of a control system can be 
identified in the control design and whether the intended functions of each element 


are conducive to the achievement of the control objective. 


Where control design is evaluated as adequate, the auditor would proceed with tests 
relating to the operation of the control and its effectiveness. Inadequately designed 
controls would be tested in terms of the cause of the noted deficiency and its 


corresponding effects. 


Are the Controls Operating as Designed? 


The auditor's initial review of the design of controls involves the gathering of 
evidence relating to management's intentions of how the controls should operate. 
Once a satisfactory understanding of the intended control design is achieved, the 
auditor should accumulate evidence which proves whether or not controls, in fact, 


are operating as intended. 


Are the Controls Effective? 


In Section Three, the criteria for assessing control effectiveness were discussed. 

To reiterate, the auditor's primary concern at this stage of the evaluation is to 
determine whether the control objectives recognized by management are appropriate 
given the control needs of the underlying system and whether established control 


objectives are in fact being achieved. 
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Audit Evalution and Major Work Instruments 


It is beyond the scope of this chapter to discuss in detail the mechanisms of 
conducting an audit assignment. (Please refer to Part 1 of this Handbook, "The 
Internal Assignment Process" for such a detailed discussion.) It is illuminating, 
however, to integrate the audit approach discussed to this point with the 
conventional work instruments associated with audit assignments. Specifically, a 
brief reconciliation of the use of the Internal Control Questionnaire and the Audit 


Verification Program with this chapter's audit approach is needed. 


Essentially, in performing an audit assignment the descriptive organizational model 
and the predetermined control model would be first defined in the review phase. 

At the outset of the evaluation phase, the auditor makes the initial comparison of 
the two models to determine whether all essential controls exist and whether they 
are properly designed. To make such a comparison, it is suggested that the narrative 
statements of the predetermined control model be converted to question format. 
The sum of questions relating to the existence and design of controls would be the 


Internal Control Questionnaire. 


Auditors determine from management and the previously prepared descriptive 
model, answers to the questions posed on the Internal Control Questionnaire. Based 


on these answers, audit verification programs are devised. 


Audit programs specify the procedures to be employed when gathering sufficient, 
valid and relevant evidence necessary to support audit conclusions about the adequacy 
of controls. Essentially, audit programs can be designed according to two formats. 
Where deficiencies arise in the existence or design of controls, audit programs are 
established to determine the causes and effects of such deficiencies. Where existence 
and design of controls appear adequate on the basis of applying the Internal Control 
Questionnaire, the audit program is designed to further test the operation and 
effectiveness of controls. Should deficiencies be noted in controls as a result of 
performing these latter audit program tests, the auditor would perform additional 


audit tests on the causes and effects of these noted weaknesses. 


The use of audit work instruments as described above is illustrated in Figure 12. 
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Summary 


Section Four focuses on the use of the descriptive and prescriptive models developed 
in Section Three. These models provide the basis of the auditor's assessment of the 
adequacy of controls. Prescriptive statements about what controls should exist and 
how they are to be designed are compared to descriptions of what is actually 


occurring within the auditee's operations. 
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CONCLUSION 


Few auditors would dispute that an understanding of the concept of control and its 
application to organizational activities are central to an effective audit process. 
While agreement exists as to the importance of control, an explanation of what it 
is and how it is to be audited is generally not available in the literature written for 


public sector auditors. 


The lack of information on auditing controls may have contributed to the existence 
of a number of different internal audit approaches to the evaluation of the manage- 
ment control framework. While a certain amount of experimentation in determining 
an approach to auditing controls is necessary given the newness of broad-scope 
auditing in the public sector, eventually audit practices must be reconciled to a 
valid conceptual basis if auditing is to retain its identity as a credible and useful 
function. This document has been prepared to provide the initial reconciliation of 
control theory to broad-scope auditing. To the extent that it has achieved this 
purpose, it should serve as a basis for deriving a consensus in the internal audit 


community as to what it means to audit controls. 
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CHAPTER THREE 


MANAGEMENT CONTROL: CONCEPTS AND PRACTICES 


INTRODUCTION 


The purpose of this chapter is to describe the relationship between management, 


particularly management control, and modern (broad-scoped) internal auditing. 


The material presented here builds on existing, well known management theory and 
practice and attempts to extrapolate it to the control design and audit domains. It 
builds a bridge between management's controlling function and general control 


theory, and between management control and modern internal auditing. 


Although controlling is generally recognized as one of the components of the 
management process, there is somewhat less agreement on what management control 


actually is. 


This chapter will begin by discussing the importance of management control to 

both managers and auditors. It will then continue with a discussion of both 
management control as an entity and controlling as a process, in terms considered 
most useful to the practices of management and auditors alike; the various roles of 
management will be discussed in terms of their respective contribution to management 
control; the dimensions of management will then be explored; and finally, approaches 


to auditing management processes and results will be elaborated. 
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SECTION ONE: MANAGERS, AUDITORS AND MANAGEMENT CONTROL 


In order to set the context for the discussion of management control concepts it is 
necessary to establish, at the outset, why control is important to both managers 


and auditors and what it is about controls that we most need to be aware of. 


The Importance of Control 


Management literature generally recognizes control as an essential role or function 

of managers or management respectively (see Table 1). However, there is less 
general agreement on what control includes, except that it has to do with ensuring 
that goals or objectives set by the organization are achieved. Since a manager's 
success or failure is typically judged by the degree to which organizational objectives/ 
goals are achieved, it is clear that whatever form control takes it is of crucial 


importance to management. 
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Table | 


A Representative List of Management Functions! 


Dale (1969) 


Planning 
Organizing 
Staffing 
Direction 
Control 
Innovation 
Representation 


Greenwood (1965) 


Planning 

Decision Making 
Organizing 

Staffing 

Direction and Leadership 
Controlling 


Gross (1968) 


Decision Making 
Communicating 
Planning 
Activating 
Evaluating 


Johnson, Kast, and Rosenzweig (1967) 


Planning 
Organizing 
Control 
Communication 


Carleson (1960) 


Planning 

Organizing and Staffing 
Direction 

Leadership 
Coordination 
Controlling 


Koontz and O'Donnell (1980) 


Planning 
Organizing 
Staffing 
Leading 
Controlling 


Longenecker (1964) 


Planning 

Organizing 

Directing and Motivating 
Controlling 


Massie (1964) 


Decision Making 
Organizing 
Staffing 
Planning 
Controlling 
Communicating 
Directing 


Mintzberg (1973) 


(Roles of Chief Executive) 
Interpersonal 
Figurehead 
Leader 
Liaison 
Informational 
Monitor 
Disseminator 
Spokesman 
Decisional 
Entrepreneur 
Disturbance handler 
Resource allocator 
Negotiator 


1 Augmented list. From Miner, John B. The Management Process - pheory 
Research and Practice, New York: The MacMillan Company, 1973, p. 48. 
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Table | (cont'd) 


A Representative List of Management Functions“ 


Drucker (1974) Fayol (1929) 
Planning Planning 
Organizing Organizing 
Integrating Commanding 
Measuring Coordinating 

Controlling 


Voich and Wren 
Newman, Summer, and Warrent 


Planning 

Organizing Organizing 
Controlling Planning 
Administer ing , Leading 


Controlling 


2 Ibid 
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The role of the internal auditor is to provide advice to management on the 
performance of all its major or significant managed operations.? > 4 The internal 


auditor then is interested in control from three points of view: 


® As an adviser to management, on the performance of its organization, 
it is desirable that the auditor be able to see the world through the 
auditee's (i.e. the manager's) eyes. Since control is the manager's window 


on the organization's performance the auditor will wish to keep that 
perspective in mind; 


e The auditor is expected to comment on the state of the auditee's control 


aye It follows that the auditors’ superiors will, presumably, 


framework. 
be evaluating the auditors on how well they perform that crucial activity. 


Control then, is of interest to the auditor as a determinant, i.e. as one 


(not necessarily the only) measure of the auditor's effectiveness; and 


® The auditor as a manager of audit operations, is also concerned with 
performing the audit task in the most efficient way possible. Given 
that we define control in a way that makes it visible as a distinct entity 
capable of being characterized, and therefore evaluated, it will be 
demonstrated that an audit is most efficiently performed by auditing 
through the control framework. As in the case of external auditing, 
detailed (substantive) testing takes the auditor beyond the examination 
of controls, into the examination of the operations being controlled and 
of the results being achieved. However, as in the case of external 
auditing, much time and effort may be saved by evaluating the framework 


of essential controls before deciding on the extent of testing required. 


3 Treasury Board of Canada, Standards for Internal Audit in the Government of 
Canada (see Standards | & 2). 


4 The Institute of Internal Auditors, Standards for the Professional Practice of 
Internal Auditing (see Section 300, Scope of Work). 
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To summarize, control in general and management control in particular are clearly 
of considerable interest and concern to both managers and auditors. To managers, 

it is an important means through which they attain and maintain economic, efficient 
and effective organizations; and to auditors, it is a means of gaining perspective, 

as the main subject of their audit activity and as means which facilitates performing 


efficient audits. 
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SECTION TWO: DEFINING CONTROL 

Given the myriad of definitions of control that abound in management and audit 
literature, it is clear that we cannot display a generally accepted version to which 
we could merely subscribe. The next best alternative is to adopt or synthesize a 
definition that is easiest to use by the two main classes of players in this discussion, 
namely managers and auditors. 


Possible Definitions 


In deciding on the most appropriate definition of control there are two (among 


many) provided in Webster's Dictionary” which are worth noting: 
1. To exercise authority over; direct; command; and 
P4, An instrument or apparatus to regulate a mechanism. 


The foregoing definitions are of interest from two points of view: 


a they suggest that both human and non-human components may be involved; 
and 
e they imply that control is not simply an activity carried out by components 


which are there for some other purpose (e.g. product/service/program 
delivery), but rather may have unique components which are exclusive 
to the control itself; i.e. that a control is capable of being distinguished 


from the entity being controlled (i.e. differentiable). 


5 Webster's New World Dictionary, Second Edition, 1976. 
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If a useful definition for control is to be adopted, it is necessary that both the 
exercise and the existence of control (both human and non-human components) be 
differentiable from the entity being controlled, in some manner. Given that control 
is differentiable, it is further necessary to demonstrate that the exercise of 
differentiation is worth the effort. The existence of human components does not 
change the nature of the discussion of control but does add considerably to its 


complexity. 


In the following, the differentiability of both control and its exercise will be 


demonstrated, as will their usefulness as distinct entities/activities. 
The Differentiability of Control 


In any discussion of the differentiability of control, the most logical place to begin 
is where there is the most agreement, i.e. the exercise of control. As already noted, 
there is general agreement that one of the major roles/functions/activities of a 


manager is controlling - the exercise of control. 


This control activity or process generally consists of: measuring or detecting the 
state of the entity that is being controlled; comparing the results with some 
reference point or performance indicator (desired state: standard, specification, 
plan, etc.); noting the deviation, if any; analyzing the deviation for size (Is it greater 
than the threshold for corrective action?) and nature (Is the deviation one of 
inadequate/inappropriate performance or inadequate/inappropriate reference point?); 
determining what action, if any, needs to be taken to correct the deviation; and, 


taking the necessary action. 


The next issue to be dealt with is whether there are components or elements of the 
organization that can be identified as the instruments through which control activities 
are exercised, and further, whether they are sufficiently distinct to be identified as 


a sub-system of the host organization. 
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For those operations/delivery systems that have automatic controls built in (e.g. in 
EDP systems or a numerically controlled manufacturing machine), there is little 
difficulty in identifying the elements of the relevant control system®, (See 
Figures | & 2 for examples of open and closed generic control systems, in pictorial 
form respectively). This is true even if some of the components serve more than 
one purpose (i.e. both delivery system and control system requirements), simply 
because they are physically identifiable and subject to characterization in terms of 
their elements and relationships. It follows that if they can be identified and 


characterized, they can also be evaluated for existence and for their effectiveness. 


Those control activities that are performed by humans (of most interest to us are 
those performed by managers) are more difficult to identify and characterize because 
some, or all, of the control process is performed by the same control mechanism, 


namely the manager. 


In this case, not all the control elements are readily differentiable from each other, 
nor are the control activities readily differentiable from the other managerial 
activities, except perhaps in terms of output. However, to the degree that the 
manager relies on other mechanisms for some elements of the control system (e.g. 
on some machine, to detect errors; on a management information system to assemble 
and present reports, which compare actual to desired performance and indicate 
deviations) at least some of its elements and their corresponding operations can be 


separately identified, characterized and evaluated. 


The more complex the organization being managed, the more managers resort to 
external mechanisms for carrying out their function. This typically consists of 
delegation of responsibilities, along with associated authority, to subordinate 
managers and using more formal management planning, organizing, directing and 
controlling methods and techniques (e.g. corporate policies; specified planning 
processes which result in formal plans; formal systems and procedures; and, formal 


management information systems). 


6 Volume II, Part 2, Chapter 2, "Control: Concepts and Applications for Internal 
Auditors" for a detailed discussion of control concepts. 
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The more complex the organization (e.g. larger; more complex delivery systems; 
more decentralized) the more complex, and therefore differentiable, its control 

systems. However, except in the simplest organization (e.g. sole proprietor) it is 
likely that at least some elements of the control system will be differentiable, 


even in relatively small and uncomplicated organizations. 


In summary both the exercise of control and the control system are differentiable 
in most cases of interest to both managers and auditors (taking into account that 
the degree of interest or concern is proportional to the risk of not accomplishing 
the results intended). The next, and most crucial, test for deciding whether or not 
to focus on differentiated control systems is the test of utility, i.e. is it worth the 


effort? 
Is a Differentiated Control Useful? 


Anyone who has studied, designed, or used control systems extensively readily 
recognizes that what constitutes a control system (i.e. its components and their 
relationships) is a function of an individual's perspective. Since managers often 
exercise their roles, including controlling, at more than one level and across a number 
of processes (both operational and managerial), and since they are generally aware 

of controls in terms of their physical manifestation, rather than their conceptual 
nature, it is not surprising that many of them are hard pressed to give one definitive 
definition of a control or control system. What exists in any medium to large 
organization is a control framework. Given that there is a division of labour between 
support groups (e.g. finance, personnel, EDP) who develop and/or operate some 
portions of the framework, and line managers, the individual manager may not even 


be aware of all the components of that framework. 


In what follows, arguments will be made to persuade the reader that a differentiated 
control system is useful to: managers, who use controls; systems and procedure 


designers/developers, who develop controls; and, auditors, who evaluate controls. 
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As demonstrated in Chapter De, Part 2, Volume II of this Handbook, organizations 

may be portrayed as systems. This form of portrayal is useful, to managers/designers/ 
auditors, as it allows the resultant model (system) to: be characterized fully, in 

terms of components and their relationships; analyzed (i.e. disaggregated from 

macro to micro level) for thorough study and decision-making; or, synthesized (from 
components into sub-systems, and then into systems), using a variety of well-known 


systems techniques. 


Good systems analysis/synthesis practice requires that systems be capable of 
assembly from, or disaggregation into, components/sub-systems and their 
corresponding relationships, which are capable of being related to their purpose. It 
follows then, that identifying control sub-systems which are relatable, and therefore 
evaluable, according to their purpose (i.e. to control objectives) is a useful 
endeavour. What is also important to note is that these systems analysis/synthesis 
processes are equally valuable to managers, designers and evaluators (including 
auditors). More will be said later about how use is made of systems techniques by 


those three classes of users. 


We next turn our attention to the issue of perspective. At the highest organizational 
level, the non-operating investor (i.e. someone who invests capital but does not 
participate in management of the firm), might view the whole organization as a 
control mechanism (see Figure 3). This control mechanism detects what return on 
investment (ROI) the portfolio of investments is contributing, compares it with the 
required ROI, analyzes the situation, determines whether that investment component 
(revenue producing organization) is likely to deliver the desired return in the future 


and retains or drops it from the investment portfolio accordingly. 


In this case the organization itself is the lowest-level systems component that the 
user is interested in because he/she does not make decisions about any of its sub- 


systems. 


7 Volume II, Part 2, Chapter 2, "Control: Concepts and Applications for Internal 
Auditors". 
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Decisions concerning the management of the organization as a whole are left to 
the Board of Directors and senior management. A useful disaggregation at this 
level would be such that the components of the corporate-level control system are 
segregated from the corporate-level delivery system, as shown on the bottom left- 


hand side of Figure 3. 


For an organization under design, this facilitates management's design process in 
that it allows management to focus on both the delivery and control systems 
independently and to align the control process with the desired level of decision- 
making and with the nature and complexity of the delivery system being controlled. 
The segregation of the control components from the delivery system components is 
particularly advantageous at the design stage as it enables decision-making about 
the most important portions of the system, the delivery system (i.e. the raison 
d'étre of the organization) to be disengaged from decisions about peripheral issues, 
which include control, at least at the conceptual design stage. This in turn allows 
for the independent determination of the design criteria for control in terms of the 
type and level of need, based on the risk considerations associated, as opposed to 
being buried in the design criteria of the system being controlled. For example, a 
large, complex delivery system may or may not need large, costly controls, depending 
on what level of risk the control system is expected to prevent or identify. The 
reverse situation is also possible. In the cases where the control is embedded in the 
controlled system it is still useful to be able to identify control components, or 


their dual role in case of shared components. 


At the operating stage, it is useful to be able to identify the contro! components of 
the organization's systems in order to be better able to pin-point problem areas and 


to take remedial action on only those components requiring it. 


As may be seen from Figure 42) the process of disaggregation can be used repeatedly 
as often as required in order to identify and characterize progressively lower level 
components of delivery systems and their associated control sub-systems for the 
benefit of managers, designers and evaluators/auditors operating at those levels of 


detail. 
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MODELLING USING THE TECHNIQUE OF PROGRESSIVE ELABORATION 
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Definition According to Perspective 


In the literature (e.g. accounting, auditing, management) control systems are referred 
to variously as internal control, financial control, management control, operational 
control, EDP control, production control, etc. In most cases the apparent inconsis- 


tencies in the above terminology can be explained in terms of the perspective taken. 


To an external auditor in the private sector, referring to the controls of interest as 
financial controls would seem to be an unwarranted and unnecessary restriction. 
After all, at the corporate level of the organization particularly for revenue 
producing organizations, financial control is internal control. However, at levels 
below the corporate level one might very well want to distinguish between financial- 
internal controls, production-internal controls, etc. Systems developers might wish 
to distinguish between EDP controls and manual controls and when describing the 
specific area of EDP controls one might wish to distinguish between general controls, 
software controls, hardware controls, data controls, etc. It is all a matter of 


perspective. 


For the purpose of this discussion internal control will be used as the generic term 
for all organizational controls. In order to focus on, and distinguish between, 
operations (production and support delivery systems) and their management, two 
corresponding terms will be used to denote associated controls, namely operational 
and managerial controls or control systems. These two types of control systems 
will be generally treated as sub-systems of the operations or management systems 
that they control, such that the controlled systems along with their control sub- 


systems together constitute complete delivery systems. 


Of particular concern in this chapter will be the management system and its associate 
management control system, however, as will be seen in the following two sections 
it is difficult to carry on any detailed discussion of management, or of management 
controls, without also discussing the operations being managed and their associated 


controls. 
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In summary, control sub-systems are differentiable from the delivery systems that 
they control, at least in conceptual terms. Furthermore, the ability to differentiate 
is equally useful to managers, advisers to managers (such as systems analysts/ 


designers) and to evaluators (including auditors). 


Finally, the apparent confusion over terminology, associated with control systems 
literature, is more apparent than real. Ultimately, there is one control framework. 
This framework or system, which will be termed internal control, may be sub-divided 
or elaborated in a number of ways, depending on the focus of attention desired, i.e. 
on perspective. For our purposes, we will be interested in the nature of and relation- 
ship between, two main components of internal control, namely, management 


control and operations control. 
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SECTION THREE: THE RELATIONSHIP OF THE FUNCTIONS OF MANAGEMENT 
TO CONTROL 


The essential theme of the following discussion will be that, although the various 
roles of management all contribute to control, there are important reasons for 
making distinctions between the contro! function and the others. The distinction is 
along two dimensions. The first distinguishes between the functions of management 
as they impinge on accomplishment of the main purpose of the organization and the 
additional elements of those functions that contribute to the control function. The 
second dimension deals with the distinction between the terms "control" (the entity 


or system) and "controlling" (the exercise of control). 
The Functions of Management 


The Functions of Management are described in a number of ways (see Table | in 
Section One), however, they can be boiled down to three essential components: 
planning (to do), implementing (doing) and controlling (determining if what was 


planned, was in fact done), (see Figure 5). 


These three essential components can be sub-divided and elaborated in a number of 
ways, depending on one's specific interest or concern. This accounts for the variety 
of views of management portrayed in Table 1. On the other hand, it is readily seen 
that all of the functions, typically attributed to the practice of management can be 
reconciled with the three-element model of Figure 5. The complicating factor, 


however, is that some functions, e.g. leadership, apply to all three. 


How the three-element mode! is sub-divided into components should be determined 


by what portrayal will be most suitable and useful for the purpose of the sub-division. 
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In this case, the purpose is to facilitate discussion of auditing the management 
process. Therefore, the sub-division employed in this chapter is such that 
components are: readily distinguished in terms of controllability; are inclusive and 
mutually exclusive; and, are amenable to systematic, progressive elaboration into 
sub-components for purposes of detailed analysis. Although the sub-division is 
performed for the benefit of evaluators/auditors, managers and control systems 
designers will find it equally useful, as their interest in effective control is, or 
should be, at least as strong as that of the auditor. Also, in order to minimize 
confusion, the detailed model used in this chapter is reconciled with the model 


presented in Treasury Board's "Principles For The Management ig 


In the planning stage, although planning for individual development oriented projects 
and for on-going operations may start out independently, the resulting plans are 
ultimately consolidated into one corporate plan. Therefore, the planning element 


is not sub-divided at this level of representation (see Figure 6). 


In discussing the implementing stage, recognition is given to the fact that there are 
two broad classes of implementation; one which represents the on-going management 
of existing delivery systems (operations; both processes and structures) and a second 
one which represents activities associated with the management of delivery system 


development and installation. 


The implementing function is sufficiently different for those two broad classes of 
activity that it is worth distinguishing them at the outset. There are several reasons 
for this. First, the development process, typically used in the organizing function, 

is expected to be carried out in ways that are explicitly (particularly in large 
organizations) distinct from the production processes employed for on-going 
operations. Second, the skill set required for these two classes of activity are 

quite different. Finally, the literature often mixes the two, thus losing the 
opportunity to explicitly recognize that those two classes of activity operate by 


different rules. 


9 Treasury Board of Canada, Principles for the Management of the Public 
Service of Canada. 
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To the degree that the development process is different from the production processes 
of on-going operations, the associated controls, and therefore the controlling 
functions, will also be different. This is because controls are normally designed to 


match the nature, extent and risk potential of the process being controlled. 


For the above reasons, the management of development activities and on-going 


activities will be portrayed (see Figure 7) and discussed separately. 


The foregoing discussion clarifies one function commonly found in management 
literature, namely, "organizing". This leaves a number of others unaccounted for, 

as leading, directing, motivating, coordinating, communicating, negotiating, decision- 
making, staffing, allocating resources, innovating, representing, activating, 
commanding, integrating, measuring, and evaluating. Since this list is multi-level 
and overlapping it will be helpful to structure it such that they can all be taken 

into account in the most efficient fashion. The structure suggested is displayed in 


Figure 8 and Table 2 and/or elaborated in Table 3. 


As may be seen from Table 3, when the common management functions/activities 
are displayed linearly, i.e. in consecutive order, the resultant list reconciles 
completely with that of the Treasury Board's "Principles for the Management..." 
document. !? However, for purposes of facilitating the discussion of management 
control, the matrix form of display in Table 2 is more informative and will be used 


in the following sections. 


—_——— —— 


10 Ibid. 
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Table 2 


Management Functions: their Relationship 
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Table 3 


Management Functions: Enumeration of Components 


Basic Functions Unique Components 


1. Planning . Strategic planning 
Operational planning 
- long term 
- current year 
Budgeting 


2. Implementing 


2.1 Organizing . Organization development 
Delivery system development 
- legislation 
- policies 
- systems and procedures 
Acquiring resources 
- staffing 
- contracting 
- purchasing 
Allocating resources 
Developing resources 


2.2 Operating . Operating delivery systems 
Maintaining delivery systems 


3. Controlling . Detecting and measuring performance 
Comparing actual performance with 
desired performance 
Analyzing the deviation and executing 
corrective action 


Common Functions 


4, Leading/Directing . Motivating 
Guiding 
Commanding 
Activating 
Coordinating 
Integrating 
Negotiating 


5. Communicating 


6. Decision-making 
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The Relationship of Management Functions to Controls 


The management functions that contribute most to controls as entities (systems) 
are planning and implementation (specifically organizing). The general relationship 
can be best portrayed in tabular form (see Table 4). No distinction is made at this 
point between operational and management controls as managers have responsibility 


for the development of both. 


The planning function's most important contributions are in two areas, planning for 
the control system (control elements and the linkages between them) and identification 
of the decision rules for performance criteria, deviation selection criteria and 


decision rules for corrective action. 


The two areas of concern are not independent, as the nature and extent of the 
criteria or decision rules chosen will influence the design of the control components 
that they populate. For example, a comparator that is expected to react to only 
large deviations (has a very tolerant selection threshold) will not have to be as 
sensitive or delicate as one that is expected to identify fine distinctions between 


actual and desired performance. 


As a more specific example, a comparator that detects deviations at the dollar 
level is more sensitive than one that only reacts to deviations over ten thousand 
dollars, in a budget control system. In the former, the system will have to store 
data in dollars and cents in order to be able to round deviations to the nearest dollar 
for comparison purposes; in the latter case, data need be stored only in more macro 


form, that is the nearest thousand dollars. 


The control element most extensively influenced by the nature and extent of decision 
rules is the activator element. This is because there are usually a number of possible 
causes for any one deviation. If the system is expected to react to material 
deviations in automatic or standard ways then the corrective mechanisms may 
become quite numerous and complex. An example of this is the numerous error 


routines typically designed and built into EDP-based systems. 
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Table 4 


Relationship between Management Functions 
and Controls 


Management Functions 
Control Elements Planning Implementing (Organizing) 


. Reference - Goals, objectives, 
Point (RP) targets, standards, 
performance 
indicators 
Detector (D) - Plans for development - Development and installation of 
of an appropriate measurement mechanisms 


measurement device 


Comparator - Plans for development - Development and installation of 
(C) of a comparator, means of comparing actual 
including a decision performance to desired 
rule for what performance 
constitutes a material 
deviation 
Activator (A) - Plans for development  - Development and installation of 
of the activator, decision rules for dealing 
including decision with deviations (analysis, 
rules for corrective corrective action) 
action 
Information - Plans for linking the - Development of the feedback 


System control elements path (information system) 
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As may be seen from Table 3, the organizing sub-function operationalizes the plans 
for the required controls systems. Specifically, it sees to the acquisition and 
allocation of appropriate resources, and to the design, development, construction, 
testing, conversion and introduction of the planned systems. It also operates them 
until such time as they are turned over to operating personnel, at which time they 


become integrated into on-going operations (delivery systems). 


Although not shown in Table 3, the organizing (development) process is subject to 
controls that ensure that it is carried out in the manner desired, and also produces 
the results desired by management, as would be done for any other process. In 
planning and developing control systems, the common management functions are 
exercised in ways appropriate to the nature of the activities being carried out. In 
this case the requirement for leadership/direction is generally more technically 
oriented (e.g. EDP, public administration, law) and specific as to area of endeavour 
(e.g. systems design, policy development, development of legislation) than in the 
usual on-going operations. Taking the skill level and object of attention into account, 
the activities, particularly the people carrying them out, need the usual motivation, 
guidance, direction, coordination, etc. Similarly, communication, as always, will 


play as important a role as it does in any human endeavour. 


The Relationship Between Management Functions and Controlling 


Controlling, as a function, is the process or activity of using the control system, to 
the degree that it exists, to detect and analyze deviations of actual from desired 
performance and to take corrective action. The contribution of other managerial 


functions to controlling is displayed in Table 5. 


There are two broad categories of controlling activities; those executed by managers 
and those by non-managerial staff. The two categories of controlling parallel the 


two categories of controls, namely managerial and operational. 


Although this chapter deals primarily with management control it is not possible to 
segregate the discussion completely as managers participate in both categories of 


controlling activity. 
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Table 5 


The Relationship Between Management Functions and Controlling 


Management Functions Controlling Activities 
Controlling - The main process of using the control system 


to detect deviation of actual from desired 
performance, analyzing the nature and extent 
of the deviation, determining and executing 


corrective action 


Decision-making - Deciding on timing and frequency of controlling 


activity 


- Deciding on importance of the deviation 


- Deciding on corrective action 


Leading/Directing - Executing controlling activities 


Communicating - Used while executing controlling activities 


Internal Audit Handbook 
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Operational! controls are generally executed by non-managerial staff, except for 
those cases where the deviation detected is beyond the scope or capability of the 
control. In the latter situation the problem is referred to higher authority, namely 
management. The level of management eventually involved will depend on the 
gravity of the problem. If a new decision rule is required, it may be supplied by 
operational management. A more serious problem, involving either the control 
system or the delivery system being controlled may be referred to progressively 
higher levels of management until a level is found with the responsibility and 


authority to deal with it. 


In the case of managerial controls, the manager executes all the steps that are not 
automated. There are two aspects of managerial control, however, that complicate 
the issue. One is the fact that much of the day-to-day managerial control is 
exercised ad hoc and informally; that is, some or all of the control system's elements 
are indistinguishable from the manager. In this case the control process may or 

may not be visible at the time of execution and is typically not visible after the 
fact. The results of the exercise of control are more likely to be visible to an 
observer only if the decision was made that corrective action is necessary, and if 


the corrective action decided upon and executed is external to the manager. 


The second aspect has to do with the distinction between the existence of control 
mechanisms and the exercise of control. It is possible for a control system to be 
dormant, and largely invisible, if no material deviations occur. In this latter case 
the control would only become visible if it is desired that the control act asa 


deterrent. 


To summarize, it is not the existence but rather the exercise of control that leads 
to either an aggressive organizational atmosphere, for employees at one extreme, 
or an anarchistic one at the other. It is mostly management style that determines 
the nature and extent of the exercise of control. Given this, any objections that 

are raised regarding the existence of controls are largely misdirected, unless they 
have to do with onerous automatic controls. Generally, when control is aggressive 


it is a case of overzealous application. 
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As discussed above, how control is viewed is very much dependent on how it is 
applied. How control is applied depends, in turn, on the purpose for which it is 


used. 


Aside from correcting unwanted deviations in performance, control when applied to 
employees has a number of derivative uses which have mostly to do with behaviour 
modification. This includes such purposes as informing, preventing/deterring, 


motivating, evaluating and training. Each of these will be elaborated in the following. 


Informing 


In the absence of general communication, or as an exception routine, the detection 
of a material deviation in actual from desired performance is often used as an 
appropriate occasion to inform the employee about preferred activities, appropriate 
processes or requirements. Although this is not considered a desirable substitute 
for properly informing employees of general, work-related requirements or events, 
prior to execution, it may be appropriate for those cases where those requirements 
or events are selectively applicable or for employees which, for some reason, did 
not receive the general communication (e.g. absent due to vacation or sick leave) 
or did not understand it when received. In this case the control function merges 


with the communicating function. 


Preventing/Deterring 


The purpose here is to prevent an event from occuring or to deter undesirable 
activity. In the case of preventing as a purpose, the control simply acts in the 
capacity of one of the two major classes of control, i.e. preemptive and corrective. 
The second purpose, deterrance, depends on close coordination with the 
communication function for effectiveness as the employee should be aware of the 
existence of the control and of the repercussions of undesirable behaviour, when 
detected. 
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Motivating 


This purpose is very much related to the others in that all control actions tend to 
affect motivation. However, how the control activity is carried out will determine 


whether the effect on employee behaviour will be positive, negative or neutral. 


In cases where the encouragement of positive behaviour is an important objective, 
care would have to be taken to present the case to the employee in terms of 
preferred future behaviour (e.g. ... a good start in the desired direction but needs 
some improvement in this or that area...) rather than in punitive terms (e.g. You 


were wrong; never do that again or else?). 


Evaluating 


Evaluation or appraisal of employees has implications for control and controlling at 
two levels. First, systemic controls, although aimed at evaluation of delivery 
systems, are useable to evaluate indirectly the employees that operate those systems. 
In order to be able to do this, the controls intended for such use will have to be at a 
detailed enough level for that to be feasible. Also, controlling action will have to 


take into account the dual role of the control. 


Second, some of the controls will have to be specifically designed for personnel 


management purposes. Management by Objectives (MBO) is an example of this. 


Training 


To the degree that all employees learn from their mistakes, all controls and associated 
controlling activities contribute to the training purpose. When training is carried 
out for the specific purpose of controlling activity, however, the way in which it is 


exercised is important. 


For example, in addition to presenting required actions to the employee in ways 
that will positively motivate future action, the content of the advice or instruction 
will need to include suggestions which will improve the ways in which future action 


will be carried out. 
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SECTION FOUR: THE DIMENSIONS OF MANAGEMENT CONTROL 
Control of Operations 


Any discussion of organizational control must necessarily begin with control of 
operations - the raison d'étre of the organization. The manager's role with respect 
to operations controls is at two levels. In the first instance, the manager is respon- 
sible for the development of the infrastructure, the wherewithall to produce the 
product or service which is the main reason for the existence of the organization 
that the manager manages. Secondly, the manager operates the control system on 


an on-going basis. 


As described in earlier sections, the infrastructure consists of static (e.g. the 
organization structure, associated delegation documents, job descriptions, etc.) and 
dynamic elements (the delivery process) and includes their respective controls. 
Figure 1 (see Section Two) displays an elementary delivery system (operating system) 
along with its associated control sub-system. In this case the control sub-system is 

a closed system; i.e. it neither receives input from nor sends output to the 


environment. 


In this scenario, it is assumed that any deviation from desired performance that is 
detected can be corrected automatically, or at least without recourse to outside 
help. This is typical of both mechnical/electronic automatic controls and of 
controls which are operated by non-managerial staff, based on pre-determined 


rules. These are the simplest form of control. 


The next scenario, portrayed in Figure 2 (see Section Two), takes account of those 
cases where the detected deviations from desired performance are such that no 
pre-determined rule exists for their automatic correction. This is necessarily an 
open control sub-system since there has to be intervention from outside the system 


(higher authority - the manager) in order to rectify the problem identified. 
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The intervention required will depend on the nature and extent of the deviation. 
The situation may require a new decision rule, new criteria or a redesign of the 


delivery system, or control system, or both. 


In both the open and closed versions of control sub-system the control may be 
exercised over the input, the process or the output (Figures | and 2 display the 
control loop related to the output only). A more thorough discussion of the subject 


of control is provided in Chapter ieee 


In summary, the manager's role with regard to operations is at two levels. First, at 
inception, the manager is responsible for the development of the infrastructure 
which will enable the host organization to produce the product or service it is 
mandated to produce, including the associated control sub-system(s). Second, during 
the operating period, the manager is required to monitor the infrastructure, 
primarily through the controls developed, and take remedial action when deviations 
of unanticipated extent or nature are detected. As indicated in the previous 
sections, these two roles may be referred to as the manager's organizing and 


operating roles respectively. 
Management Controls 


The foregoing dealt with control of simple operations, those which have only one 
process or activity. In this case operations level (i.e. delivery system) controls and 
managerial (i.e. results) controls, although distinct conceptually, can be physically 
identical (see Tables 6 & 7). 


In what follows, conditions will be described that will require some distinctions to 


be made between operations and managerial controls. 


The conditions that require distinctions between delivery system and results 
controls are of two types. The first has to do with complex operations and the 
second with the management role at levels higher than direct operations 


management. 


el Volume II, Part 2, Chapter 2, "Control Concepts and Applications for Internal 
Auditors". 
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Complex Processes or Activities 


Complex activities can be of two kinds, those that simply number more than one, 
i.e, the manager has several to manage, and those that produce a product or service 


that is not simply the sum of the individual activities (see Figure 9). 


In the case of the first kind, the results have to be aggregated or consolidated for 
reporting purposes. Although it is conceivable that they could be a simple tabulation 
of the results of each activity, generally, it would not be efficient for the manager 
to keep track of them individually and it would certainly not be acceptable to relay 


such a detailed tabulation to higher-level management. 


The usual method used for aggregation or consolidation of results for purposes of 
efficient reporting (control) is to use performance indicators (i.e. indexes of perfor- 
mance which represent the detail). An example of this for economy controls is the 
reporting of budget totals (actual vs. budgeted), instead of details, to higher 


management. 


The second kind is more complex than the first in that the composite results are 


not simply an aggregation of the results of individual activities. 


An example of this, again for economy controls, is where the number of activities 

is sufficient to warrant a common support group, thus achieving economies of scale. 
Here one could contemplate simply aggregating individual activity results as before, 
however, since the activities are not homogeneous (e.g. a mixture of line and 
support) the control system would, at the minimum, at least report aggregate 


results (totals) for the two main types of activities separately. 
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Table 6 


Typical Control Objectives 


A) Delivery System Control Objectives 


Input Control Conversion Output Control 
Objectives (Transformation Process) Objectives 
Control Objectives 


Controls to ensure: Controls to ensure: Controls to ensure: 
- Selection and - Prevention or detection of - Completeness of 
maintenance of accidental errors in output 
suitable infra- conversion of input to 
structure, resources output - Accuracy or 
and raw materials freedom from 
- Prevention or detection of error in outputs 
- Appropriate handling fraudulent activities 
of demands for during the conversion of - Timeliness and 
service input to output appropriate 
distribution 
- Security of conversion of output 
process to ensure 
continuous operation - Compliance with 
specifications 
- Compliance with authorities from higher 
and prescribed policies and authority 
procedures 
B) Results Control Objectives 
Economy Control Efficiency Control Effectiveness 


Objectives 


Controls to ensure 


that resource 
acquisition and 
maintenance is 


done economically 


Objectives 


Controls to ensure 
that output achieved 
for input applied is 
optimal 


Control Objectives 


Controls to ensure 
that the entity's 
output is according 
to plans and has 
the desired effects 


& 
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Table 7 


Example of Combined Delivery System and Results Controls 


Delivery System 
(operations) 


Conversion Process 


Input(s) Output(s) 
Operations (Delivery System) Controls 
Input Control: Process Control: Output Control: 
input qty./qual. process operated output specifications 
meets specifications? as prescribed? met/not met? 
Managerial (Results) Controls 
Economy Control: Efficiency Control: Effectiveness Control: 
actual input process operated actual results 
agrees with efficiently: i.e. agree with planned 
budgeted input ($)? actual output + input results? 
agrees with planned (quantity, quality, 


level of efficiency delivery, etc.) 
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COMPLEX PROCESSES OR ACTIVITIES E 


(a) Aggregation of results 


Conversion Processes 


Input(s) Output(s) 


(b) Integration of results 


Conversion Processes 


~ 
° 
{) 
° 
(_) 


(s 


Figure 9 
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Another example, this time for effectiveness controls, could be where the finished 
product is an assembly of the outputs of a number of other activities. In this case, 
again, the activities are not homogeneous, i.e. the final output is not simply the 
sum of the individual activity outputs. Also, effectiveness is not simply a reflection 
of the physical output of the process(es) but rather of the effect the combination 

of its production and delivery has on the recipient in particular and the environment 
in general. This is particularly applicable to intermediate outputs. A specific 
example would be a training activity. Its direct output is a trained individual, 
however, its intended effect is to achieve better performance of that individual on 
the job. Effectiveness controls are meant to cover not only direct and intermediate 


outputs, but effects as well. 


Higher-level Management 


Controls systems for higher levels of management are inherently complex because 
the systems they control are complex. This is because the system being controlled 
includes all lower-level systems. For example, second-level managers manage their 
first-level manager(s), usually more than one, along with their respective operations. 
This implies the control of at least two types of processes, the operations, at least 
at the aggregate level, and the managerial process of the subordinate manager(s) 


(i.e. planning, organizing, etc.). 


The hierarchical nature of the management process, including management controls 
is displayed in generic form in Figure 4 (see Section Two) and an elaboration which 


includes the components of the management process in Figure 10. 


If the components of the management process are treated like any other process, 
conceptually, then all discussion thus far concerning delivery system controls applies. 
The managerial processes have inputs, conversion processes and outputs, like any 
other process, and may have controls for these three elements accordingly. The 
decision to implement formal controls would be made on the same basis as any 

other control decision, i.e. based on materiality, risk and cost criteria. As usual, 

the cost of the control must be justified by the risk avoided. This does not preclude 


ad hoc, informal control being exercised. 
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MANAGEMENT PROCESS CONTROL MODEL 


To Higher Authority 


Control system 


The Management 
Process 
(subordinate Mgr.) 


Input 


Figure 10(a) 


Control system 


The Management Process 
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Controlling d 
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Plans for 
Consolidation 


Communicating 
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mgt., clients, etc. 
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I 


Demands Operations ! Output 


= —— <p Products/services 


Resources 


Figure 10(b) Legend 
D = Detector 
RP = Reference Point 
C = Comparator 
A = Activator 


Figure 10 
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Since the operations of all higher-level management include at least two types of 
activity (i.e. product/service delivery systems and their management) they are 
necessarily classed as complex. Therefore, the discussion in the preceding part, 
regarding the distinction between operations and managerial controls is generally 
applicable here. The main difference, due to the typically hierarchical nature of 
the managerial structure, is that managerial controls will tend to be progressively 


more aggregative for progressively higher levels of management. 


Where management has adopted the Management By Objectives (MBO) system of 
performance measurement (contro!) for individual managers there may exist two 
parallel control systems, one for the organizational unit (e.g. responsibility centre) 
and one for the manager (the MBO system). It is, of course, possible to merge the 
two, however, it must be remembered that the criteria/indicators for individual 
and unit performance are not necessarily identical. This may dictate independent 
controls. 
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SECTION FIVE: MANAGEMENT AUDITING 
Results vs. Infrastructure (Structure and Process) Auditing 


The results vs. compliance auditing issue has many facets. Some of the controversy 
surrounding this dichotomy is as much due to misunderstanding of the terminology 


as to real differences. 


First, let us deal with the compliance issue. The usual use of the term compliance 
audit has its roots in private sector, external auditing (financial attest auditing), 
where it is used in relation to testing of internal controls in order to make judgments 
about the extent of substantive testing required. In this case, the compliance 
testing activity plays a role which is secondary to the main role of financial attest 


auditing. 


A second root is the public sector external audit role of determining compliance 
with authorities. Again this compliance testing role is largely subsidiary to the 
usually more significant roles of financial attest and, more recently, economy, 


efficiency and effectiveness audits. 


In both the above cases the roles of substantive testing and compliance testing 


occupy primary and secondary roles respectively. 


In internai auditing the roles of compliance and substantive testing are reversed. 
Internal audit's main raison d'étre is to reflect to management the degree to which 
results/performance planned or prescribed are in fact being achieved (i.e. the degree 
to which actual performance complies with required performance). This is done by 
reviewing and assessing internal controls. In this case the substantive testing process 
is similar in content but different in role. Its role is a supporting one to the 
compliance one, aimed at substantiating assertions about actual vs. required 
performance of delivery systems (structures and processes) and their respective 
results; and, at identifying causes of non-compliance and consequent effects or 


impact on the managers operations and environment. 
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As may be seen from the above, internal auditing, although technically compliance 
auditing (in external audit terms), has as its target all aspects of compliance, 
including: compliance of the existence and operations of infrastructure (structures 
and delivery processes) with requirements (e.g. legislation, policies, regulations, 
guidelines, systems specifications, procedures, etc.); and, compliance of results/ 
output with plans or requirements (e.g. output specifications, performance indicators, 


and economy, efficiency and effectiveness objectives, i.e. the 3Es). 


This allows the auditor to provide management with opinions on whether desired 
results are being achieved and on whether those results are likely to continue to be 


achieved. 


In summary, although what internal audit does is compliance testing, technically (in 
external audit terms) the assertions that it reviews and assesses are in terms of 


both "infrastructure" and "results". 
Auditing Operations 


As may be seen from the discussion in Section Four, in the case of simple operations, 
infrastructure (structures, processes) controls and results (management) controls 
might be physically merged. In this case there may be as few as three physical 
controls but supporting as many as six assertions, representing six control objectives 


(see Figure 11). 


The presence of formal, physical controls and the formal statements of corresponding 
assertions about the state of the respective organizational elements are determined 


by the risks involved. 


In auditing management controls, and providing opinions on the associated assertions 
about the state of the operations, the above described possibility has to be taken 
into account. In other words, unless the respective risks are not sufficient to 
warrant it, the auditor is expected to verify assertions about all six basic control 
elements (i.e. input, conversion, output; the 3Es) even though there may be less 


than six controls to examine. 
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THE RELATIONSHIP BETWEEN CONTROLS AND ASSERTIONS 


(Simple operations) 


OPERATIONS 


INPUT OUTPUT 


INFRASTRUCTURE 


INFRASTRUCTURE 


CONTROLS 


ASSERTIONS INPUT INFRASTRUCTURE OUTPUT 


ECONOMY EFFICIENCY EFFECTIVENESS 


Figure 11 
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In the case of more complex operations, there will generally be at least six controls 
(again, given that the risks involved warrant them); however, there may be many 
more, depending on how many activities/processes are included in the operations 
(see Figure 12 for two possible configurations; Cases A and B). In addition to at 
least one control for each of the results/management assertions (the 3Es) there 


will be input, conversion and output controls for each major process or activity. 
Auditing the Management Infrastructure 


In conceptual terms, the components of the management process may be treated in 
the same way as any other operation; i.e. each management activity or component 
may be broken down into the elements: input, conversion and output. As well, any 
decision to develop formal controls should be based on risk criteria as would be the 


case for any other control decision. 


As with all operations, management activities or components may be broken down 
into sub-components for purposes of more specific management attention (e.g. the 
planning component might be broken down into strategic long-term operational 
and current-year operational planning and resource planning). If this results in 
associated system and sub-system controls, they would of course be included in the 
auditor's pre-determined control framework for examination and assessment 


accordingly. 


Since management is generally associated with operations being managed, a pre- 
determined control framework for any level of organization above the first level 

(i.e. lowest level responsibility centre) would automatically qualify as a complex 
operation. The minimum set of operations in this case would consist of a delivery 
system being managed and its associated managed structures or processes (see 

Figure 13). Here one would expect to find at least nine controls in the predetermined 
control framework), six for delivery systems and three for results (Figure 12), 
although there may be more. The reason for a minimum of six controls for delivery 
systems is that the delivery systems for management operations and production 
operations are sufficiently different that they cannot be combined, as were the 


activity 1 and 2 controls in Case A, Figure 12. 
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THE RELATIONSHIP BETWEEN CONTROLS AND ASSERTIONS 


(Complex operations; more than one activity) 


OPERATIONS 


ACTIVITY 1 


STRUCTURE; OUTPUT 
PROCESS 


OUTPUT 


ACTIVITY 2 


STRUCTURE; OUTPUT 


PROCESS 


CASE A 


COMBINED OPERATIONS 


INPUT STRUCTURE; PROCESS OUTPUT 


CONTROLS 


INPUT STRUCTURE; PROCESS OUTPUT 
ASSERTIONS 
ECONOMY EFFICIENCY EFFECTIVENESS 
CASE B 


ACTIVITY 1 ee ae ACTIVITY 2 
CONTROLS 
STRUCTURE; STRUCTURE; 
PROCESS OUTPUT INPUT PROCESS OUTPUT 


INPUT - OUTPUT INFRA- OUTPUT 
STRUCTURE STRUCTURE 
ASSERTIONS 
ECONOMY | EFFICIENCY | EFFECTIVE- |] ECONOMY] EFFICIENCY | EFFECTIVE- 


Figure 12 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 3, Section 5 - 301 - 


THE RELATIONSHIP BETWEEN CONTROLS AND ASSERTIONS 
(Complex operations; operations and their management) 
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In summary, from the point of view of the development of a pre-determined contro] 
model, the process of auditing an operation (whether simple or complex), along 

with its associated management structures and processes, would be treated as a 
complex operation overall. However, unlike the case of two homogeneous 
operations, in this case a minimum of six rather than three delivery system 

controls are required due to the substantial differences between management 
structures/processes and the structures and processes which comprise the operations 


being managed. 


Auditing Results (Economy, Efficiency and Effectiveness) 


The Relationship between Economy, Efficiency, Effectiveness and Results 


The Standards! define "economy", "efficiency" and "effectiveness" as independent 


and mutually exclusive terms as follows: 


Economy: "Economy refers to the terms and conditions under which the government 
acquires human and material resources. An economical operation acquires those 


resources in appropriate quality and quantity at the lowest cost." 


Efficiency: "Efficiency refers to the relationship between goods or services 
produced and resources used to produce them. An efficient operation produces the 
maximum output of a specified quality or characteristic for any given set of 
resource inputs or it has minimum inputs for any given quantity and quality of 


service provided." 


Operational Effectiveness: "The extent to which operations achieve their objectives 


or goals." 


Program Effectiveness: "Program effectiveness concerns the extent to which a 


program achieves its goals or other intended effects." 


12. Treasury Board of Canada, Standards for Internal Audit in the Government of 
Canada. 
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As may be seen from the foregoing definitions, although the terms are distinct, they 
are not independent. The definition for economy overlaps with that of efficiency 

in that part of the efficiency equation (i.e. ratio of output achieved to input/resources 
used) absorbs resources used, the main concern of economy. In fact the term 
efficiency, taken in its global sense (i.e. all resources used in an operation) could 


be, and often is, interpreted as including economy. 


Similarly, efficiency is related to effectiveness in that efficiency relates output to 
input either by minimizing input while keeping output constant or increasing output 
for a constant input or a combination of the two. In all cases, output (the prime 


concern of effectiveness) plays an important role. 


Finally, if effectiveness is interpretated globally (i.e. all objectives and goals 
achieved) it could readily be seen as encompassing economy and efficiency as they 


are managerial goals or objectives as well. 


As will be seen later, regardless of their non-independence, the three terms are 


useful to both managers and auditors for a variety of reasons. 


The term "results", if interpreted from the point of view of the manager, at whatever 
level, is a very broad term. To the chief executive officer/deputy head it could 

mean aggregate economy, efficiency and effectiveness in terms of: organizational/ 
program outputs and effects, achievement of organizational policies (whether output 
or infrastructural); achievement of organizational profitability or economy/efficiency 
targets, achievement of quality standards, achievement of good labour relations, 

etc. To the manager of a delivery system (whether program delivery or support) it 
may mean economic, efficient and effective operation of that system; to an activity 
manager it may mean delivery of intermediate output of a specified quality, at 


specified times, to specific users. 


In general, the manager has two broad, results oriented concerns: meeting output 
goals/objectives (quantity, quality, timing, distribution) and maintaining the ability 
to continue to meet these goals/objectives (i.e. the delivery system) within the 
framework of organizational (e.g. head office) and environmental (legal, regulatory, 


etc.) requirements or constraints. 
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Auditing Economy 


As indicated in the definition, economy has to do with acquisition of resources of 
appropriate quality and quantity at the lowest cost. In global terms this could be 
interpreted as the acquisition, and maintenance in readiness, the ability or capacity 
to produce the product or service which is the raison d'étre of the organization or 


organization unit concerned. 


Given that interpretation, it would then include both raw resources (labour, materials) 
and delivery systems (structures, processes/systems and procedures). This would 


roughly parallel the "organizing" role of management. 


The most recognizable example of an economy control is a budget vs. actual report, 
which compares input planned vs. input used, in dollar terms, (in the production of 

a given quantity and quality of output).* Although the budget report is a well-known 
control mechanism, it is seldom associated with the conditional phrase shown in 
brackets above, at least not explicitly. Yet it is only in that context that it is truly 


meaningful. “ 


An exception to that rule would be the case of discretionary spending, where the 
manager is being committed to maintaining a certain level of capability rather 
than producing a predefined quantity of goods or services. However, even in this 
case some relationship between input and output might be established, perhaps on a 
statistical basis, if the output has some homogeneity. For example, although in 
handling a legal case the time or effort to complete any one case may be unpredic- 
table, if the legal adviser handles only cases of a certain type, which have a limited 
range of complexity it may be possible to establish the average number of cases 
one lawyer can handle per year, within a predictable precision range, 


e.g. 150 cases +5. 


This number can be used for purposes of determining the number of lawyers required 
to handle a legal practice consisting of a predicted number of cases per year. In 

the case of development of a system the economy control could be in project 
management terms, such as time estimated vs. time taken and/or dollars of expen- 


diture estimated vs. used. Again the implicit assumption is that there is an implied... %« 
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"for the amount of work accomplished" condition attached, in terms of milestones 


achieved or output produced. 


In auditing economy, the auditor expects to find economy performance indicators 
for all major units of resources acquired or maintained, related to given units of 
outputs produced or capability maintained. Mathematically this could be expressed 


as: 


Ey = Ca 
Cp 
Where: 
EY = Economy Indicator * 
C. = Cost of Input actually used* 
SF = Cost of Input planned* 
3 These variables may be summations of a number of resources, the only 


limitation being that they have to be in the same units (e.g. $s) 


Where resources cannot be readily correlated to output produced, before the fact, 


the economy indicator has two possible uses: 


& As an indicator of expenditure management control (i.e. dollars spent 


vs. dollars planned); 


8 As an indicator of forecasting ability (actual units processed, necessitating 
actual dollars spent vs. estimated units of demand, requiring planned 
dollars of expenditure). Where resources can be related to output, 
efficiency is a better measure of performance than economy. In fact, 
in this case, economy performance is absorbed by the efficiency perfor- 


mance measure. 
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Auditing Efficiency 


Efficiency is a term that is used in a number of ways. For example, it is often used 
interchangeably with the term productivity, a general term connoting abundance - 
not necessarily for the least possible input, but used somewhat differently in business 
circles. At the other end of the scale, it is used in a much more specific manner by 
industrial engineers to indicate the degree to which the ratio of actual output over 


input meets a standard output over input ratio. 


The more usual use of the term is simply: 


Ee = O where E, = Efficiency 
2 T Fa 
] ] = Input 
O = Output 


This is also what is usually meant by productivity. 


In this chapter, the two terms will be used interchangeably, in order to conform to 
general usage, however, two gradations of the use of the ratio will be presented: 

one representing the case where actual efficiency is compared to planned, which is 
based on history; and the second, based on an engineered index of efficiency, where 


actual efficiency is compared to an engineered standard. 


(a) Case 1: Actual efficiency compared to planned efficiency 


In this case planned efficiency may be arrived at in a number of ways. The most 
usual approach is to base it on historical data. Other possible ways are, comparison 
with other similar activities in the same organization, other organizations, other 


sectors (public with private and vice versa), industry-wide data, etc. 
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Ways in which such efficiency/productivity ratios might be expressed are: 


(i) 


(ii) 


(iii) 


(iv) 


Where: 


Note: 


Ic 


As with the economy indicator, efficiency indicators and their 
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Ea; where Ea = O (actual) and Ep = O (planned) 
Ep I ] 


Ip (for constant output) 


Oa 
Op (for constant input), 
(ice. Oa /Gp. =  Oamnaxa mic 
Ic/ Ic Ic Op 


Ta 
Tp (for constant output/input) 


Efficiency index 


Actual efficiency achieved 


Planned efficiency 


Input 


Input held constant 


Output 


Actual time taken (hrs., wks, mos. yrs.) 


Time planned 


Qa 


Op 


components may be summations or aggregates provided that they are 


expressed in the same units. 
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(b) Case 2: Actual efficiency index achieved compared to a standard index 


In this case the actual ratio or index of output over input achieved is compared to a 
standard efficiency ratio, which was arrived at through an engineered study such as 


time and motion study or work sampling. 


It follows, from the foregoing discussion, that efficiency controls depend on knowing 
both the input and output components of the index. However, it is possible to use 
the efficiency indicator as a control where the output cannot be precisely defined 

if there is some assurance that the output has remained approximately constant 


over the period concerned. 


This type of indicator is generally less precise; its precision being dependant on the 
degree to which output has indeed remained constant. In any case, this type of 
efficiency indicator is less reliable than those for which all components are precisely 


defined. 


Efficiency performance indicators may be employed by managers at all levels from 
individual indicators at the project/activity level, to aggregate indicators at the 
global level (e.g. program or organization-wide), depending on how significant 


management considers those projects, activities, etc. to be. 


In auditing for efficiency, the auditor expects to find efficiency controls for all 
major or significant activities (whether projects or on-going activities), as part of 
the control framework. Where such controls are not in evidence it will be necessary 
for the auditor to demonstrate the feasibility of establishing such a performance 
measure and the benefits to be derived from doing so, before recommending that 


such a contro! be installed. 


Auditing Effectiveness 


Auditing effectiveness has a number of dimensions, depending on level of focus and 
the definition used. At the minimum, a distinction is made between output effective- 
ness and program effectiveness. In the federal government, output effectiveness is 
the domain of internal audit while program effectiveness is assessed by the program 


evaluation function. 
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This does not mean, however, that internal audit has no part to play in program 
effectiveness control. Internal auditors would expect to find program effectiveness 
performance indicators as part of the organizational control framework. The 
difference in treatment occurs in the substantive/detailed testing phase of the 

audit. Where output effectiveness indicators are missing or ineffective, the auditor 
would be expected to detect and substantiate this, and to demonstrate the feasibility 
and cost-effectiveness of installing appropriate controls. In the case of program 
effectiveness controls, the auditor would be expected to detect their lack or 
ineffectiveness but would defer to program evaluators to substantiate the ineffective- 


ness and to demonstrate the feasibility and cost effectiveness of appropriate controls. 


Output effectiveness can of course be measured at all levels of activity, starting 
with projects and progressing through various intermediate product or service levels 
to final product or service, i.e. the product or service that is delivered to some 


client outside the organization under audit. 


The general expression of the effectiveness indicator is in terms of a ratio of output 


achieved to output planned, or to a standard (e.g. a specification). 


In mathematical terms, this may be expressed as: 


FE. = Oa 
Op 
Where E, = Effectiveness Index 
oF = Actual Output 
oe = Planned or Standard Output 


As with economy and efficiency indicators of performance, the effectiveness indicator 
may be a summation or aggregation of a number of either homogeneous outputs or 
homogeneous indicators or it may simply be an array or list of a number of indicators 


which are not readily combined (i.e. non-homogeneous). 
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CONCLUSION 


This chapter has described and demonstrated the importance of control as a 
management mechanism in terms of its structure (the control system or framework) 
and its process (the operation of the control system). Control was also discussed in 
terms of its value to the auditor as a proxy measure for actual organizational 


performance and as a method for achieving an efficient audit process. 


In order to clarify the distinction between controls and the systems they control, a 
set of definitions were developed which are expected to be equally useful to 


managers, systems designers and auditors. 


Since management's role is crucial to the existence and operation of control, ina 
number of ways, this relationship was discussed in some depth. This discussion 
supplemented by a discussion of the dimensions of controls and control activity 


then set the stage for a discussion of management auditing. 


The discussion of management auditing is capped by a discussion of auditing economy, 
efficiency and effectiveness, an area of activity which is responsible for considerable 


confusion in the literature. 


The treatment of management control in this chapter is not viewed as definitive in 
any sense, but it is hoped that it serves to clarify some of the relevant issues and 


clear up some misconceptions for managers systems designers and auditors alike. 


om 
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CHAPTER FOUR 
ANALYSIS CONCEPTS AND PRACTICES FOR INTERNAL AUDITING 
INTRODUCTION 


In its early stages, the main activity of internal auditing consisted of "the verification 
of the authenticity and validity of charges", and dealt primarily with accounting 
and financial controls. One of the prime concerns of the auditor was the detection 


and prevention of fraud.! 


Currently, more emphasis is placed on a constructive approach to internal auditing, 
with at least as much interest shown in reviewing and evaluating the effectiveness 
of internal controls, so as to eliminate waste and inefficiency and improve producti- 
vity, as in those controls maintained for financial accountability and the detection 


and prevention of fraud. 


For example, the scope of the Internal Audit Standards” requires the auditor to 


review the following areas: 


The scope of internal audit shall encompass all aspects of a department's 


operations. The internal auditor assesses and expresses an opinion upon: 


® the design, development, implementation, and operation of all systems, 


procedures, processes and controls, including computer-based systems; 


@ the reliability and adequacy of information available for decision-making 


and for accountability; 


e the extent to which available information is utilized in the decision- 


making process; 


1 See Standards for Internal Audit in the Government of Canada, Treasury 
Board of Canada, Comptroller General, 1982. 
2 Ibid, p. 24. 
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« the adequacy of protection afforded public funds and assets; and 
& the extent of compliance with legislative, central agency and 


departmental directions. 


This modern concept of internal auditing has seen a shift in emphasis of audit 
coverage and substance from "dealing primarily with accounting and financial 
matters" to "any area of the entire organization where protective and constructive 


; 3 
service to management can be provided". 


Figure | illustrates this changing emphasis and responsibility of internal auditors. 
This changing emphasis brings with it a need for a corresponding expansion of audit 
concepts and techniques in order to maintain audit rigour in these new areas of 


audit activity. 


The overall purpose of this chapter of the Handbook is to provide an overview of 

the theory and conceptual context of a variety of analysis techniques currently 

used in adjacent disciplines but equally applicable to internal auditing. The thrust 

of the coverage provided in this chapter is to go beyond current conventional audit 
literature and practice (e.g. Internal Control Questionnaires; flow charting; statistical 
sampling), borrowing those analysis methods and techniques from adjacent disciplines 
(such as management science, operations research, economics, industrial engineering, 
etc.) which have the potential for improving the rigour and depth of the internal 

audit process. Since the subject matter presented here is already well covered in 

the literature (see the copious references in the Appendices) it is not discussed in 


detail; rather, the intent is simply to increase the auditor's awareness. 


The methods and techniques described in this chapter fall into two broad categories: 
those which are immediately applicable by the average auditor without help and 
those which will require the help of a specialist and which may need further develop- 


ment in order to adapt them to the internal audit environment. 


3 Ibid, p. 2. 
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Section One of this chapter will describe analysis concepts currently employed in 


other disciplines that are applicable to the field of internal auditing. 


Section Two will further describe the application of these analysis practices with 
specific reference to the main activities inherent in the internal audit process. A 
detailed list of references, which highlights pertinent source material for the 
selected techniques and relates them to the audit process, is presented in the 


appendices (see Table of Contents). 
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CHANGING EMPHASIS AND RESPONSIBILITY 
OF INTERNAL AUDITORS 
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SECTION ONE: ANALYSIS CONCEPTS 


Analysis Concepts in the Context of the Internal Audit Process 


While agreement exists among auditors as to the importance of control, an 
explanation of what it is and how it is to be audited is generally not available in the 
literature written for public sector internal auditors. Nor is it dealt with in any 
other audit literature (private or public sector; external or internal audit) except 
for coverage of the subject of internal control, the scope of which is generally 


restricted to financial controls. 


Volume II, Part 2, Chapter 2 of this Handbook, "Control: Concepts and Applications for 
Internal Auditors" provides an initial reconciliation of control theory with broad- 

scope auditing. Chapter 2, together with Chapter 3, 'Management Control: Concepts 
and Practices", describes how the objectives and scope of the audit function can be 


met through the review and evaluation of the organization's control framework. 


An organization's control systems can be represented by various models or 
frameworks. One useful model, designated the Descriptive Organizational Model 
(DOM), is a framework for systematically organizing the vast amount of detailed 
auditee information that has been collected. This model enables the auditor to 
describe the audit entity without concern, at that point in time, for judging the 


adequacy of the processes being reviewed, and to focus on "what is going on". 


A second useful model, designated the Normative or Predetermined Control Model 
(PCM), is a prescriptive organizational model that provides the auditor with a 
comparative basis for determining the adequacy of the auditee's control framework. 
Two levels of control are addressed by this PCM framework. The first identifies 

the general management controls in place to ensure due regard for economy, 
efficiency and effectiveness; the second identifies the generally applicable process 
controls in place for regulating the day-to-day operations of the audit entity. This 
model focuses primarily on what controls should be in place (i.e. control objectives). 
It also contains the standards or criteria through which the actual auditee operations 


are to be evaluated. 
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The organizational control model serves as a vital tool in the structuring and 
communication of an auditor's approach to an examination. For example, this model 
will be of value in developing control questionnaires or checklists for use in the 
subsequent phases of the audit. Figure 2 illustrates the context of such models in 
the audit process, which is characterized by the major phases: assignment planning, 
review, evaluation, verification and reporting. For example, the review phase 
explicitly represents the structure and operation of the entity under review with a 
Descriptive Organizational Model, and explicitly represents the controls that should 
be operating with a Predetermined Control Model (see sub-components II.1 and II.4 


respectively in Figure 2). 


The process of auditing controls is increasingly being applied by the internal auditor 
to managerial and organizational concerns (as illustrated by Figure i);7 This 
transition from financial control to more broadly based managerial control is 


depicted in Figure 3 from the perspective of controls. 


Given this change in emphasis and scope, auditors must develop an understanding of 

a broader range of control strategies, frameworks and corresponding analysis methods 
if they are to properly assess operations controls and controls for economy, efficiency 
and effectiveness. Different auditee operating systems will require different 
controls. In turn, these requirements will present the auditor with a range of issues 
or problems in attempting to model the entity (e.g. boundary/closure problems; 
levels of systems/variable depth; adaptive/corrective systems; biased and/or 


simplistic judgments). 


In order to address these concerns, structured analysis frameworks from a number 
of adjacent disciplines may be used to provide an appropriate balance between 
auditor and auditee judgment and structured evaluation. The following section will 


expand on the use of analysis concepts (and techniques) in auditing. 


4 Ibid, Page 1. 
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OVERVIEW OF THE CONTROL CONTEXT FOR THE AUDIT PROCESS 


AUDITING 


AUDIT 
PROCESS 


CONTROLS 


MANAGEMENT 


INTERNAL ACCOUNTING 


FINANCIAL ACCOUNTING CONTROLS 


CONTROLS 


CONTROLS 


Figure 3 
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Elaboration of Analysis Concepts 


Inherent in the use of organizational modelling, analysis methods and techniques 

for internal auditing is the need for more and better-grounded analysis and synthesis 
tools. Currently, the main techniques employed by the internal auditor are a carry- 
over from the financial audit perspective. These include document analysis, various 
interview techniques (e.g. Internal Control Questionnaires) and various verification 
techniques such as cross-checking, recalculation, reconciliation and non-statistical 
sampling. On occasion, more advanced techniques such as: macro-level organiza- 
tional modelling (e.g. ASB's "Egg" model)? , flow charting, statistical sampling, and 


regression analysis have been used. 


A number of other relevant models, methodologies and analysis techniques are 
available from adjacent disciplines (e.g. management science, operations research, 
survey sampling, industrial engineering, etc.) that could improve the arsenal of 
audit techniques, thus increasing the rigour and laterally the impact or worth of 


the internal audit function. 


For example, in the recent book by White, M.J. (et al), Managing Public Systems: 
Analytic Techniques for Public Administration, a number of mainstream analytic 
techniques for building and assessing a management system are presented (see 

Table 1). Many of these techniques have direct application to the current role of 


internal auditing. 


Table 2 illustrates how the general classes of analysis concepts and techniques 
interface with the audit process. Section Two will highlight a number of these 


analysis techniques and describe how they may be applied to internal auditing. 


5 From: "A Structured Methodology for the Conduct of Comprehensive Auditing", 
Audit Services Bureau, Supply and Services Canada, 1982. 
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In using or adapting any of these more sophisticated analysis methods, the auditor 
will typically employ some form of systems theory, but in a control setting. For 
example, in a particular systems theory application to internal auditing, the 
emphasis of the investigation may be on questions, assertions or opinions as well as 


on answers. 


The use of general systems theory facilitates the exploring and examining of various 
strategies to serve the same end (e.g. analysis techniques for the review stage, 
verification techniques) or ultimate outcome of the audit itself (e.g. opinion on 


controls, opinion on infrastructure or opinion on actual results). 
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Table | 


Analysis Techniques for Building and Assessing 


Modelling 
_Concepts 


@ Environmental 


management models 


® Flowcharting 
concepts and 


methods 


® Network scheduling 
models (PERT, CPM) 


® Forecasting models 


(Delphi, Spread- 
Sheet/S-curve) 


a Management System 


Design and 


_____Decision-making _ 


Rational and 
incremental models 
of managerial 


decision-making 


Decision analysis 
models (Bayesian 
Analysis, Sensitivity 
Analysis, Decision 


Tree Analysis) 


Operations research 
methods (Goal 
Programming, 
Queuing, Replace- 
ment Planning, 


Simulation) 


Evaluation 


Forecasting for 
operational control 
(Smoothing Techniques, 


Disaggregation Techniques) 


Program evaluation 
methods (Performance, 
Strategy Analysis, 
Outcome Evaluation, 


Policy Analysis) 


Organizational accounting 
systems (Management 
Systems Models and 
Methods) 


6 Adapted from: White, M.J. et al, Managing Public Systems: Analytic 
Techniques for Public Administration, 1980. 
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SECTION TWO: ANALYSIS PRACTICES 


Cross-referencing the Internal Audit Process with Analysis 
Techniques 


The audit process is displayed in its traditional form in Figure 2. The dynamics of 
this process is better demonstrated, however, in Figure 4, where the mainstream is 
assignment planning, evaluation and reporting in that order, with review and 


verification playing supporting roles to evaluation. 


The assignment planning phase of the audit is a derivative of long-term and annual 
planning and a prerequisite to the conduct of the audit of a specific audit assignment. 
It starts with familiarization with the audit entity and culminates in a detailed 

audit planning memorandum. The plans may, of course, need to be refined at any 


stage of the audit, as shown in Figure 4. 


The principal products of the assignment planning phase are: 


& development and definition of audit purpose, scope, and objectives; 
® preliminary identification of significant audit areas and development of 
audit approach (lines of inquiry); 


communication of audit purpose, scope, objectives, and approach; 


identification of staff and other resources required and assignment of 
responsibilities; 

preparation of time, budget and work schedule; 

planning of audit working papers; 


preparation and approval of an assignment planning memorandum; and 


briefing for auditee management. 
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DYNAMICS OF THE AUDIT PROCESS 


ASSIGNMENT PLANNING 


VERIFICATION 


REPORTING 


Figure 4 
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The review phase of the audit is devoted to the gathering of more detailed 
information concerning the entity's programs, objectives and activities, to developing 
a model of the current state of the entity under audit, to developing a normative 
(predetermined) control model, if not available from past activity, and to limited 
testing in order to identify potential problems or other matters warranting further 


detailed examination. 


The principal products of the review phase are: 


& detailed documentation of existing management control framework, 
6 predetermined control models, and 
a potentially significant audit areas and issues. 


The evaluation phase of the audit begins during the planning of the assignment and 
continues through to the drafting of the final audit report. The assessment of the 
management control framework (e.g. against a predetermined control model) is the 
core of the audit process. This iterative process includes data verification, cause 
and effect analyses, and development of audit findings, conclusions and 


recommendations. 


The principal products of the evaluation phase are: 


register of essential controls, 

register of compensating and environmental controls, 
summary of weaknesses, 

plan for the verification work, 


analysis of causes and effects, and 


audit findings, conclusions and recommendations. 


The verification phase of the audit is designed to collect sufficient additional, 
reliable evidence to support audit findings and conclusions. It is essentially an 
adjunct to the evaluation phase, focused on substantiating and consolidating 


preliminary findings and on providing evidence for the analysis of causes and effects. 
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The principal products of the verification phase are: 


® substantiation of essential controls, 

8 verification of significant control deficiencies, inefficiencies and 
weaknesses, 

® substantiation of causes and effects, and 

& completed audit working papers. 


The reporting phase of the audit is the mechanism by which the auditor communicates 
findings to management to assist them in monitoring the economy, efficiency and 
effectiveness of internal management practices and controls, in improving the 
control framework and in ensuring adherence to established policies, plans and 
procedures. The analysis concepts and techniques being described in this chapter 

are generally not applicable to the reporting phase, therefore it will not be discussed 


beyond this point. 


The four phases -- assignment planning, review, evaluation and verification -- are 
described in detail in Volume II, Part 1, of this Handbook, "The Internal Audit 
Assignment Process". Appendix I at the end of this chapter summarizes these 
selected phases and sub-phases of the audit process by applicable, relevant analytic 
techniques (listed in Table 2). The following provides highlights of a representative 


set of these analysis techniques which are useful for auditing. 


Elaboration of Analysis Techniques 


Table 2 presents an overview of the analysis techniques from the perspective of a 
general classification of disciplines or fields of study and application. The classes 
of techniques are mainly drawn from non-auditing fields, with some overlap of 
currently used auditing methods. For example, the set of techniques under the 

first column (denoted, for simplicity, "Flowcharts/Networks/Descriptive Modelling") 
make up most of the tools commonly used by the modern internal auditor. Since 
much audit literature already exists on these techniques (see Appendix II), the 
selected techniques highlighted in the remainder of this chapter are taken from the 


remaining classes of techniques. 
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Note that in Table 2 there is a natural ordering of complexity of techniques as one 
moves from simple concepts, techniques and methods (such as flowcharts and 
networks) to more advanced, current techniques and methods. In turn, there is a 
parallel emphasis on the level of knowledge and experience required of the auditor 
for the proper use of the more advanced techniques and progressively more of a 


need to employ specialists in their application. 


The overall purpose of the following sub-sections is to highlight briefly the essential 
nature and application of the chosen technique within the audit process. Details on 
the methods themselves can be found in the appropriate references listed in the 


Appendices. 


Before proceeding with the descriptions of the selected techniques, a brief note on 
the general classes of analysis concepts and techniques is warranted. The seven 
headings in Table 2 -- Flowcharts/Networks/Descriptive Modelling; Decision Theory; 
Management Science (MS)/Operations Research Research (OR) Theory; General 
Systems Theory; Economic/Financial Analysis; Sampling Theory; and Evaluation/ 
Control Theory -- represent a rich cross-section of disciplines and domains of study 
taken mainly from non-auditing literature sources. These general classes are not 
exhaustive nor are they independent of each other. In some classes, an ascending 
hierarchy of complexity is presented in the arrangement of the techniques (e.g. in 
classes: Flowcharts and Networks; Decision Theory; General Systems Theory; 
Economic and Financial Analysis) as one moves down the particular column of the 
table. In other classes, the ordering is simply in line with the audit process phases 
(e.g. Sampling Theory; Evaluation and Control Theory). Many of the individual 
technique categories contain multiple techniques, methods and models (e.g. the 
Sampling Strategy category under the Sampling Theory class breaks into a number 
of sampling techniques appropriate for statistical and non-statistical auditing: 
physical unit sampling - attribute or variable; dollar unit sampling; etc.). Only 
selected techniques or models will be addressed in the following sub-sections, and 
within these - in most cases - only one or two individual techniques or models will 
be highlighted. In any case, all techniques have appropriate references listed in the 
relevant Appendix (II to VIII) at the end of this chapter; this source material is also 
cross-referenced to the major phases and sub-phases of the audit process in 


Appendix If 
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Decision Theory 


Issue or Problem Analysis Models 


Within the Decision Theory literature and associated management science 
communities, a number of useful models are currently being used to aid the 
designer, problem solver, decision-maker, and (analogously) the auditor, in 


addressing issue or problem definition, formulation and analysis. 


The purpose of these issue or problem analysis models is to allow the users to 
tackle the particular issue/problem in the most effective manner by first examining 
their own way of thinking or addressing the concern in the first place. (See 


Appendix III for references.) 


For example, the typical problem-solving procedure, as depicted in Figure 5, which 
instinctively leads to questionnaire designs, data collection and processing, is first 
put to its own test. A number of fundamental questions are first posed in order to 
help create a better understanding of the issues, so as to improve the solution 
process. For example, the following questions are addressed in R.S. Stainton's "The 


Question is More Important than the Answer" (Appendix III, A.1.): 


What is the problem? 

Whose problem is it? 

How will I know when I have found a solution? 

Who will benefit? 

Who will suffer? 

What are the constraints? How important are they? 
How flexible are they? 

Is there more than one objective? 

Is there a hierarchy of objectives? 


How long is the solution expected to last? 


How do I know when I have sufficient facts? 
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TYPICAL PROBLEM-SOLVING PROCEDURE 


SYMPTOMS 


PROBLEM DEFINITION 


DATA 
ANALYSIS 


SETTING OF OBJECTIVES 


DATA 
COLLECTION 


IS PROBLEM WELL 
ENOUGH DEFINED? 


SOLUTION 


Figure 5 
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e When will other factors enter into the problem? 
% In what ways may the problem be redefined? 
e How might solutions cause other problems elsewhere? 


A variant of the foregoing approach is to begin the problem-solving process from a 
dissolve perspective (i.e. addressing what ought to be), then proceed to either a 
solve (what is?) or a resolve (what can we do?) line of thinking. This problem- 
solving process (described in detail in the reference found in Appendix III, A.4) 

tends to focus on the root definition of the problem, recognizing first the perspective 
of the client (auditee). This process does not depend on data. It may lead toa 
redesign of the whole process under study. Strong interpersonal and communication 
skills are required for the typical management-level application of this technique. 

In practice, this model approach usually leads to problem-reduction, and some trade- 
off solutions in the solve (i.e. optional answers to bounded sub-problems) and dissolve 
(i.e. managerial compromise or "satisfice"” approach to solutions) components of 


the model. 


With respect to the audit process, the scope and use of these types of problem 
analysis models is broad, practical and immediate. For example, this technique can 


be applied to the following phases of the audit process: 
Assignment planning: 


6 in the development and definition of audit scope parameters (such as 
nature of auditee, audit work required and type of audit suggested) and 


audit objectives (such as criteria formulation); 


® in the determination of auditability, audit risk and materiality/ 


significance; and 


rj Herbert Simon's term for the situation where the manager takes the pragmatic 
way out and adopts a workable (good enough) solution to a problem, rather 
than attempting to optimize, (See Administrative Behavior, 2nd Edition, 
Macmillan, i957). 
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e in the communication of audit scope, objectives and approach to 
management. 


Review: 


& in the elaboration of the audit issues so as to determine, clarify and 


confirm the purpose, scope and objectives of the audit entity; and 


6 to assist the development and, in particular, the objectivity of predeter- 
mined controls (model), which form the basis for analysis and evaluation 


of the existence and effectiveness of controls. 
Evaluation: 


@ in assisting in the judgments or decisions related to the significance and 
interpretation of various effects of control weaknesses, by confirming 
these assessments with the problem formulation and solving process so 


as to present a consistent and relevant set of findings; 


® in assisting in the development of a detailed verification plan, so that 
techniques appropriate to the audit objectives and the nature of the 


audit entity are chosen; and 


i in assisting in the analysis of causes and effects of problems identified 
in the audit entity, so that audit findings and conclusions are stated 


clearly and unambiguously. 
Verification: 


€ in assisting in the further review and development of audit criteria, or 


the selection of the most appropriate method of testing; and 


e in assisting in the substantiation and verification of inefficiencies and 
weaknesses in the audit entity, particularly since these findings relate 


back to the audit issues and management concerns. 
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The use of these issue or problem analysis models and techniques should become 
part of every internal auditor's tool kit for conducting audits of any kind (e.g. 
program, function, systems, special, etc.). These techniques are easy to learn and 
highly useful and relevant to the audit process. Appropriate use should produce 
significant beneficial impact on audit results, both at a general and specific level 


of application. 
Bayesian Decision Models 


Within Decision Theory, there exists a powerful set of Bayesian modelling techniques 
that allow the auditor to rigorously address certain unknown but testable situations. 
In particular, Bayesian frameworks use a general mathematical modelling approach 
to express the entire decision space of the user's problem in terms of numerical 
quantities related to risk. These quantities, called posterior probabilities, assist 

the decision-maker in rigorously stating the solutions to the problem in terms of 

her or his a priori (subjective) knowledge of the problem as new empirical evidence 


becomes available. (See Appendix III, D for references.) 


With respect to the audit process, the Bayesian model technique would typically be 
used in the verification phase during detailed audit testing. For example, wherever 
statistical sampling is used, the Bayesian approach might serve as a complementary 
tool to the classical theory of statistical inference. It provides the auditor with a 
logical framework within which to relate both judgment and sample evidence, in 


proper proportions, to the economic consequences of possible actions. 


To outline the technique in its simplest form, suppose we wish to investigate some 
tentative audit findings during the verification phase based on a verification sample. 
Assume that the (prior) probability, based on the results of the review phase, that 
some transaction or control process is working satisfactorily is estimated as X per 
cent. Now suppose that a verification sample shows that the probability that it is 
working satisfactorily is only 1/2 of X per cent. The formal Bayesian procedure 
would calculate the new revised (posterior) probabilities based on this new evidence 
in the presence of the original information (details not shown here). Depending on 
the size and coverage of the verification sample, the auditor may have to revise 


the probability of satisfactory control significantly downward or upward. 
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Bayesian models and analyses take into account subjective probabilities and expected 
gains or losses in much the same way as they are intuitively considered by the 
managers or decision-makers. A priori judgment is important if sample information 

is meagre, as is the case in most small samples. As the sample size increases to 

the point where the evidence is overwhelming, a priori judgment may well be discarded. 
In the limiting case where a census is taken, the posterior probabilities of the 

Bayesian procedure approach those shown in the sample itself (i.e. classical sampling 
results). 


The use of Bayesian models and techniques may well have specialized but useful 
application in various verification-phase situations for the internal auditor. The 
simpler models are relatively easy to learn, particularly when illustrated with a 
variety of case studies. Appropriate use should assist the auditor in completing the 
requirements of detailed audit testing and other verification procedures for particular 


situations. 


Management Science and Operations Research Theory 


Decision Tree Analysis 


Within Management Science (MS) and Operations Research (OR) Theory, a number 

of useful types of decision analysis are currently being used by managers to solve 
complex problems. These consist of: decision tree analysis (to be highlighted below), 
probabilistic forecasting and multi-attribute analysis. The second and third types 
noted here have been developed from decision tree analysis; the former uses 
probabilistic outputs of decision analysis as inputs to the development of forecasts 
(which, in turn, can be used to support decisions about planning, investment and 
marketing, for example). The latter type evolved from decision analysis where 

there was a need to balance multiple objectives and provides a way to quantify and 
trade off the multiple factors that affect the decision, i.e. the final choice of 


alternative. 
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The purpose of decision tree analysis is to display the anatomy of sequential decision 
problems; that is, action choices available to a decision-maker (auditor) and options 
that are determined by chance. The decision tree allows for the calculation of the 
expected payoff for a series of sequential choices. (See Appendix IV, A for 


references.) 
The general decision tree structure is depicted in Figure 6. 


The construction of the tree should follow the detailed guidelines given in the 
aforementioned references, which also contain many illustrations and case studies 


of the technique. 


With respect to the audit process, the use of decision tree analysis would typically 


occur in the following phases: 


Assignment planning: for example, during the development of the audit 
workplan (so as to more accurately establish optimum final choices from 


alternative resources and scheduling mix). 


Review: to assist the auditor in the validation and assessment of data and 


systems accuracy based on a small sample. 


Verification: to assist the auditor in further developing audit procedures for 
verification testing (where minimum cost of effort or maximum benefit of 
coverage and depth of audit is sought), and to assist or perform any particular 
verification procedure (where sequential decisions or choices are made under 


uncertainty). 


The use of decision tree analysis is mainly seen as a complementary tool to other 
procedures or tests and for particular situation applications (such as in the planning 
processes to examine the allocation of resource decisions). This basic type of 
decision analysis is easy to learn, and is well documented and illustrated in the 
literature. Auditors should be able to use this technique whenever the particular 
situation arises; it supports objective and unambiguous decisions in the presence of 


multiple choices or alternatives for action. 
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GENERAL DECISION TREE STRUCTURE 
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Risk Analysis 


From MS and OR Theory, a set of techniques commonly known as risk analysis are 
increasingly being used in adjacent fields to assist analysts and managers to make 


decisions under uncertainty. 


The purpose of risk analysis in the decision-making context is to extend or elaborate 
upon the measure of value used for choosing from decision alternatives. For example, 
the value of control to managers is not a simple linear function of the amount of 

cost savings or net benefits. Essential control and the form it takes is most important, 


while successive increments of control have less and less subjective value. 


In order to assist the decision-making process in an uncertain environment, a 
quantitative measure of the decision-maker's attitude toward risk is needed. A risk 
equivalence measure, called utility, is typically used to express the decision-maker's 
aversion to risk in non-monetary units, which can more realistically represent any 
non-linearities in the judgment process. The decision criterion is then to pick the 
alternative with the highest expected utility, rather than the highest expected 


monetary value. 


The procedure for making decisions in an uncertain environment involves these 


steps: 


define the possible events that can occur; 
define the actions that can be taken; 


determine the value (in dollar or utility) of each action-event combination; 


describe the decision-maker's uncertainty about the events by a set of 
probabilities; 

find the expected value of each alternative action by multiplying its 
value for each event by the probability and summing; and 


° select that alternative with the highest expected benefit (or utility). 


To specify the above decision procedure is merely to organize the decision-making 
process in a systematic and logical fashion. All decisions made under uncertainty 
go through these steps, although some steps may be done in an intuitive manner. 


(See Appendix IV, C.l and C.2 for references.) 
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A number of similar techniques for analyzing value decisions and model implications 
(from the perspective of decision-making under uncertainty) are described in the 
references found in Appendix IV, C.3, C.4 and C.5. 


With respect to the audit process, this technique can be applied to the following 


phases: 


Review: in the initial assessment analysis of the existence and effectiveness 
of management controls, where much uncertainty and little quantitative data 


is usually the norm underlying the decisions at this stage in the audit. 


Evaluation: in analyzing the effects of perceived operational weaknesses 
expressed either in quantitative or qualitative terms, or for studying various 
hypotheses on the reason for, and the significance of, system weaknesses such 


as audit criteria deficiencies. 


Verification: in assisting the further development of audit criteria, or to 
analyze collected evidence against various decision implications (as part of 


the cause and effect analyses). 


The use of risk analysis techniques in the auditing context is evolving, particularly 
with respect to making or assisting decisions under uncertainty. These techniques 
are relatively simple to learn but much work is needed in bridging the gap to 
particular audit applications. Appropriate use should assist the auditor in more 
objectively addressing decision issues and outputs at specific points in the audit 


process. 
Market-research Models 


Within MS and OR Theory, a broad set of models - known under various names, such as 
market exploration or market analysis, or as denoted here to represent all such 
models, market research, - are used by decision-makers essentially in resolving 

the questions: which product characteristics or market strategies are most 
influential in the development and marketing of a successful product. In the auditing 


context, this might translate into "what process, structure, management or control 
characteristics are most likely to result in satisfactory operations or results?" 
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Market research is primarily concerned with the discovery and the interpretation 

of facts or patterns of facts. Information is usually assembled in various ways, for 
example by: iterative searches, statistical analysis, mail questionnaires, telephone 
interviewing, direct interviewing, and designed seminars. Interviewing itself may 

be by questionnaire, presubmitted or used as a checklist, structured or unstructured; 
of an exploratory nature; or conducted as the pilot preparation for an extended 
program. Tailored and automatic analysis methods are used to highlight data 
structures and relationships, such as conjoint analysis, multi-attribute utility analysis, 


factor analysis, regression analysis and so on. 


Marketing data can be classified into primary facts, corresponding to reasonably 
firm sequences of behaviour, based on historical data; and secondary or derived 


facts (i.e. which are much less firm, stable or objective) such as those which are: 


- frequently based on opinion, 

- often of a transitory nature, 

- usually about a contingent future, 
- vulnerable to subsequent action, 
- subject to sharp discontinuities, 

- fraught with uncertainty, and 


- seldom susceptible to proof. 


Although checklists are employed, interviewing tends to be unstructured in 
exploratory work, with information interpreted progressively, to make the most of 
any leads uncovered. Questioning is so directed as to discern the underlying facts, 
significance and reasoning. By correlation in a matrix, data which may prove 
unobtainable directly may be inferred and then confirmed. Such information serves 


as an aid in decision-making. 


Many of the market-research models have a direct link to economic and statistical 
models. In addition, statistical and stochastic process models (e.g. Markovian 
process models, Bayesian analysis models) are some of the more advanced applications 


of this approach to decision-making. (See Appendix IV, E for references.) 
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With respect to the audit process, the approach inherent in the use of these models 
can be used at various phases of the process, but is most applicable in the verification 
stage of the audit. The emphasis of this methodology is on the logical structuring 

of the gathered data (e.g. primary, secondary), confirmation of the data through 
various sources (e.g. dependent, independent or third party), and rigorous assessment 


of the data using a battery of statistical techniques. 


The use of market-research models, methods and techniques in the auditing context 

is new but has promise. Much insight into all aspects of data rigour and meaning 
could be gained by the systematic application of some of these well-tested techniques: 
they are relatively easy to learn, assume some use of statistical methods in their 

more advanced applications, and could significantly improve (in the longer term) 


the associated decision-making processes of auditors and managers. 


General Systems Theory 


Organizational Analysis Models 


Within General Systems Theory, there is a broad class of macro-oriented models 

and techniques for assisting managers in the design, development and implementation 
of better (e.g. more effective) management systems. It is a given that the ways in 
which organizations are designed and the environments in which they operate make 


a difference in organizational performance. 


One important trend in addressing the problem of organization assessment in a 
reliable and valid way is the increasing application of system frameworks or models, 
which employ measurement instruments and processes that are scientifically valid 
and practically useful for assessing organizations on an ongoing basis. (See 


Appendix V, B for references.) 


Such an approach to organization assessment has been well documented in "Measuring 
and Assessing Organizations" (Appendix V, B.1). The measurement instruments of 
this approach, denoted Organization Assessment Instruments (OAI), are highlighted 


below. 
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OAI is a set of questionnaires and survey procedures for measuring the dimensions 
included in the OA framework; that is, various characteristics of the context, 
structure, and behavior of the overall organization, work groups and jobs. The 
measures in the OAI, which have proven to give good indications of reliability (i.e. 
reproducibility of results) and validity (i.e. substance and integrity of results 
themselves in relation to the objectives), explain large variations in the efficiency, 
effectiveness, and job satisfaction of different types of organizations, work units, 
and jobs. Specific areas where improvements and revisions are needed can be 


identified with these measures. 


The basic process followed in OAI technique development is exemplified by Table 3. 
It illustrates the characteristics of questions requiring answers if we are to 


understand organization effectiveness. 


In practical terms, the OAI represent an attempt to develop an organizational 
information system within the host organization. Once developed and implemented, 
data obtained periodically with the organizational and performance information 
systems are linked together as an overall OA management information system 
(MIS). This system, in turn, when incorporated as part of a data-based management 
and organizational development process, can provide managers and analysts with 
systematic, longitudinal information for addressing basic questions about the 
performance of their organizations on the basis of how they are structured and the 


environments in which they operate. 


With respect to the audit process, the use of organizational analysis models bear 


directly on several audit process phases: 


Review: in assisting the development of predetermined control models (so 
that the resulting control model is more appropriate to the characteristics of 


the audit entity). 


Evaluation: in assisting with or complementing the evaluation of the existing 
management control framework against the predetermined model (such as 
confirming whether the existing control framework is appropriate to the 


entity from an organizational effectiveness perspective. 
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Verification: in assisting the development of more detailed audit criteria 


before proceeding with further procedural testing of controls. 


The use of organizational analysis models should be part of every internal auditor's 
arsenal of techniques, particularly for the evaluation phase. These techniques are 
easy to learn, highly useful and relevant to the audit process, but most effectively 
applied when tempered with previous experience. For example, many parallel 
developments at both a conceptual! and an applied level exist between the OA-MIS 
model and the Predetermined Control Model, as described in other parts of the 
Handbook. Appropriate use should produce a significant impact on audit results, 


primarily at a general (managerial) level of application. 
Goal Programming Models 


Within General Systems Theory, in the Research and Development management 
literature, there is a trend to more elaborate but realistic models that take a goal- 
directed, contextual and systemic perspective when dealing with organizational 
issues. (See Appendix V, C for references which provide a starting point to this 


literature.) 


The purpose of goal programming models, as defined in the preceding context, is to 
better understand the required relationship between goals, resources, structures 

and processes (e.g. for the audit entity), and incorporate this knowledge into theories 
and designs for more effective structures and processes within the organization. 
Figure 7 illustrates the organizational implications of contextual goals and generic 


factors. 


Organizational goals and their corollary - constraints - are built into decision- 

making processes which are conditioned by context (e.g. the organization's decision 
making and communication structures, environmental constraints, state-of-the-art 
technology). Goals are part of a package which includes beliefs, actions, and 
processes of goal-setting and translation into action - a package which is tied together 


and given meaning by the context. Goals and constraints are how the implications 
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of the environment show up within an organization, mediated by organizational 
processes and the members' intentions and beliefs. Goals, therefore, should conti- 
nually be examined within contexts, and contexts in relation to goals. More detail 
related to goal-profile frameworks and their application to organizational effective- 


ness issues can be found in the references found at Appendix V, C.2 and C.4. 


With respect to the audit process, the potential scope for use of this type of "goal 


programming" model is significant for several major audit functions: 


Review: in assisting the development of predetermined control models 
appropriate to the characteristics of the audit entity, or in assisting in the 
preliminary determination of potentially significant audit issues related to 


uneconomical or inefficient operations, given the auditee context. 
Evaluation: for example: 


(i) assisting in various evaluative tasks, such as answering the question 


"is the existing control framework appropriate to the entity?"; 


(ii) determining which management controls are essential to the 
effective operation of the auditee system, given the auditee goals 


and context; 


(iii) assisting in the cause and effect analysis stage, by appropriately 


contexting the issues under investigation. 


Verification: assisting in the development of more detailed audit criteria for 


additional testing of controls. 


The use of goal programming models is an advanced technique which may be used 
to assist the internal auditors in future audits. Much work still needs to be carried 
out before the technique can be properly introduced into the audit process ina 


formal manner. 
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The positive benefits for, and impacts of the technique on, subsequent audits may 

be quite substantial. For example, the more an auditee is subject to external 
influence in its goal determination, the more important it is for the auditee to 
develop inter-organizational and boundary role mechanisms that permit and facilitate 
the required two-way interaction with respect to: what is needed; when; in what 
form; what is feasible; what could be possible; and so forth. The lack of such 


mechanisms hampers an organization's ability to be responsive. 


The proper use by the auditor of goal programming models can assist the 
understanding of the auditee management mechanisms conditioned by these boundary 
constraints. This knowledge, in turn, would aid the development of reliable 
predetermined control models and associated criteria for evaluating the system's 


effectiveness. 


Economic and Financial Analysis 


Cost/Benefit Analysis 


This general term represents a range of techniques which varies in complexity from 
very simple to highly sophisticated. Internal auditors already use many of the 
simpler, less structured versions of this technique, but much more emphasis on the 
cost implications of the internal audit process and of its focus of attention, the 
auditee, is crucial to future effective auditing. For this reason this technique is 


reemphasized here. 


Within the general class of techniques denoted Economic and Financial Analysis, 
cost/benefit analysis is one of the most obviously applicable in internal auditing. 
Cost/benefit analysis refers to a procedure where the economic efficiency of an 
operation or program (e.g. audit object under examination) is determined, usually 
expressed in monetary terms as the relationship between costs and outcomes. Cost- 


effectiveness (discussed under Evaluation Control Theory) is another. 


Cost/benefit and cost-effectiveness analyses can range from simple to sophisticated 
technical procedures. In some situations, formal complete efficiency analyses are 


either impractical or unwise. For example, the required technical procedures may 
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be beyond the technical expertise of the project team, or may be unnecessary, 
given either the very minimal or the extremely high efficiencies of the operation 
or program. In addition, the practice of expressing political, ethical or other 
considerations, in economic terms regarding particular input or output measures, 
may discount the usefulness or relevancy of this type of analysis for a given 
application. Nevertheless, managerial choices between alternative modes of 
operation or competing programs are, at least in part, based on relative payoffs in 


economic terms. 


A comprehensive cost-benefit analysis requires estimates of the benefits of the 
audit object (e.g. program, function, mechanism or operation), both tangible and 
intangible, and the costs of undertaking the program action, both direct and indirect. 
Once specified, the benefits and costs are then translated into a common measure, 
usually a monetary unit. The assumptions underlying definitions and measures of 


costs and benefits strongly influence the resulting conclusions. 


Note that all the requisite data for proper cost/benefit calculations are seldom 
available. Where benefits are undefined, cost-effectiveness is the more appropriate 


technique. 


Examples of the methodology of cost/benefit analysis abound in the literature. A 


good starting point is provided by the references found in Appendix VI, D. 


The specification, measurement, and valuation of costs and benefits - procedures 
that are central to cost/benefit analysis - pose two distinct problems. First is the 
identification and measurement of all program costs and benefits and the second is 
their expression in terms of a common denominator (e.g. monetary values). A number 
of approaches have been specified for monetizing outcomes or benefits: direct 
measurement of monetary benefits; market evaluation; econometric estimation; 

use of hypothetical questions (e.g. "what if" scenarios); and so on. For the results 

of a cost/benefit analysis to be valid and reliable and to reflect fully the economic 


effects on the entity under examination, all relevant components must be included. 


With respect to the audit process, cost/benefit analysis techniques can be used in 


the following phases of the audit process: 
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Assignment Planning: in deciding on significant audit areas and on resource 
allocation. 


Evaluation: in studying the significance (in monetary units) of program 
weaknesses identified by the auditor, or in substantiating the significance of 


particular findings during the causes and effects analysis stage of the audit. 


Verification: in performing substantive testing of particular operations - 
where an economic perspective is valid and evidence on operational control 
efficiencies is called for or in determining whether it is cost effective to do 


additional verification. 


The use of cost/benefit analysis techniques is well suited to particular applications 

in the audit process, primarily for substantive testing where an economic perspective 
is valid and suggested. The more sophisticated versions of these techniques should 
only be used by experienced auditors or specialists with a background in the discipline. 
Increasing use of this technique in appropriate audit situations will lead directly to 


more effective auditing. 


Sampling Theory 


Sample Design 


Within sampling theory, a large body of literature both in the auditing and non- 
auditing fields exists on the application of statistical sampling to problem measure- 
ment, hypothesis testing (confirmation) and inference development (i.e. reliable 
and valid projection of sample findings to predetermined populations of interest). 
References in Appendix VII highlight some of the more pertinent sources for the 


auditor. 


Due to the large number of good references on this subject, the following description 
will only highlight some of the key principles associated with sample design and 


their interface with the audit process. 
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An overview of the sampling framework for the auditor is provided by Figure 8. 
The sample design stage of a study encompasses all aspects included in this figure. 
The stages denoted (sampling strategy, sampling implementation and sample 


evaluation) will be described in more detail in the following sub-sections. 


The basic principles common to most sampling plans and testing activities -whether 
they are strictly an applied, statistical approach or the traditional, for auditors, 


nonstatistical or judgmental approach - are: 


® Auditor's Objective 
- Testing consists of selecting and examining a group of items (a 
sample) to make assessments about the entire group of items (the 


population) from which they are drawn. 


@ Determination of Sample Size 
- The number of items selected should be sufficiently large so that 
there is an acceptable probability that the sample results will 
approximate the actual parameter values of the population that 


they are drawn from. 


% Method of Sample Selection 
- Sample selection should be performed on a representative basis so 
that an appropriate cross-section of the population will be 


examined. 


® Projection and Interpretation of Results 
- Conclusions drawn on the basis of a sample should recognize that 
errors discovered in a sample are likely representative of the 
errors that you would expect to find in the population, and that 
the actual extent of errors in the whole population likely falls 
within a range around the estimated extent of errors determined 


by projecting the results found in the sample. 
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AN OVERVIEW OF THE SAMPLING FRAMEWORK 
FROM THE PERSPECTIVE OF THE AUDITOR 
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The logical sequence of steps, although usually iterative in nature, for the actual 


design of a sampling plan is summarized below. (Note that these activities typically 


are completed during the planning stage of an audit.) 


Avoid early discussion of sample size.” 

e Formulate the general goals and uses of the survey. 

® Specify the frame (i.e. the entire listing of units in the population under 
study). 

a List the variables to be measured and the criteria to be used. 

® Discuss the sampling strategy (e.g. sample like and unlike groups, lightly 


or heavily; spread the sample out; create artificial groups for unknown 


segments of the population; special treatment for special cases; for 


unequally sized groups use probability proportional to size sampling). 


® Make the final sample decisions. 


e Choose the Getiatcres 


9 Important Note: 


One should avoid an early discussion of sample size, and 

choose the estimators at a latter stage in the design sequence 
because the selection of a sample size (an estimation procedure) 
requires firstly the specification of both an estimator and 

the method of sampling, except in those instances where 
information is available from previous studies, or when a 

pilot study is a possibility. For example, assuming the use 

of the sample mean and simple random sampling will result 

in a determination of sample size quite different from assuming 
the use of a ratio estimator and a cluster sample design. 

What is really needed - given the pressure of budget 
considerations for an early discussion of sample size - is a 
"sample allocation decision" that uses the budget in the 

most efficient way. This means choosing both good estimators 
and good methods of sample selection; that is, sample designs 
that have the best chance of being efficient. 
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With respect to the audit process, where sampling requirements are pre-determined, 
sample design technology plays a significant role in the assignment planning phase, 
with lesser involvement in the later phases - except as refinements to the design 
(e.g. in the review phase when carrying out a system walk-through with a small 
sample to verify the accuracy of the data collected; in the verification phase when 
choosing the most appropriate method of testing based on the established features 
of the population under study). 


For example, in the assignment planning phase, knowledge and experience with 


sample design theory and practice can: 


® assist the development and definition of audit parameters (such as 


providing solid arguments for or against a specific type of audit; 


@ assist in identifying the audit risk and the materiality or significance 
underlying potential audit approaches (i.e. for those areas of inquiry 
which seem appropriate for providing the auditor with insight (or 


evidence) into (or of) significant operations of the entity); and 


8 assist in detailed planning (e.g. resource allocations, scheduling, audit 
objectives, approach, budgeting), and communications of the audit plan 


to management. 


Where sampling requirements are not pre-determined they may be identified in the 


evaluation phase; to be carried out in the verification phase, as before. 


The use of sample design theory and practice in the planning, conduct, analysis and 
evaluation of audits should be fundamental knowledge for all internal auditors. The 
essential basics and principles of sampling are well documented and easily taught 

to beginners. More sophisticated use of design methodologies can be readily obtained 
by employing appropriate expertise on the audit team (e.g. typically, an experienced 
survey statistician). Appropriate investment in this technique will pay significant 
dividends in producing quality audits, both at a general and specific level of 


application. 
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Sampling Strategy 


The purpose of this sub-section is to further highlight the context and meaning of 


sampling strategy within the sampling theory framework. 


From the perspective of the auditor, sampling strategy is typically that stage in 
the design of the audit where the auditor must decide upon how many and what 
kind of items should be sampled. More generally, the strategy must encompass 


decisions related to: 


6 the nature or type of characteristic(s) to be sampled (e.g. attribute: 


yes or no; variable: continuous scale, expressed in dollars; or both); 


® the nature of the sampling unit (e.g. if the account of an individual is to 
be sampled, then the realm of physical unit sampling is entered - 
which is based on much different theory and application than dollar 
unit sampling, where each dollar recorded in the population is sampled 
directly regardless of the account identification at the first stage of 


selection); 


* overall sample design configuration, and the respective sample size 


alternatives and associated audit or sampling risks; 


¥ the nature of the selection process itself (e.g. statistical versus non- 


statistical based selection procedures); 


® the estimation (inference) procedures for projecting the sample results 


to the target populations; and 


© the data collection, processing and monitoring procedures (e.g. quality 


control checks) for the implementation stage of the audit. 


In addition, sampling strategy addresses many of the detailed technical issues of 
the audit during the earlier phases or sub-phases. For example, treatment for 
anomalous or highly unusual, data-dependent findings; missing, unavailable, beyond 
scope, or non-response data; and measurement or other sampling problems; is at 


least tentatively dealt with by the end of this stage. 
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With respect to the audit process, sampling strategy impacts the process in three 
distinct areas: 


Review: in the initial verification of the accuracy of the data collected, and 
the subsequent ramifications for the sample design in succeeding phases of 
the audit. 


Evaluation: in substantiating various data-dependent hypotheses of concern 
to the auditor, so as to prepare the more detailed sampling strategy for the 


subsequent verification testing of this concern. 


Verification: in communicating various sampling options (e.g. different 
methods of verification testing) to the audit manager, highlighting the potential 


ramifications of each option. 


The use of sampling strategy principles and practice in an auditing context is best 
achieved when carried out in conjunction with staff specialists. Auditors should, 
however, know the basic principles and applications of sampling strategy to ensure 


efficient and effective auditing at all levels of application. 
Sampling Implementation 


The purpose of this sub-section is to further highlight the stage in the execution of 

a sampling plan most commonly known as the implementation or execution 

stage. Other names for this activity are: data collection, fieldwork, or verification. 
The emphasis, however, will be on the techniques available to the auditor in carrying 


out the required tasks of this stage. 


Referring to Figure 8, it is seen that this stage bridges the sampling strategy and 
sample evaluation stages in the overall sample framework. Generally, the methods 


for collecting statistical data are: 


® direct observation; 
@ personal interview; 


® mail interviews; 
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° telephone interviews; and « 
8 controlled experiment (e.g. particular time and motion efficiency studies). 


Quality control procedures are typically employed in the sampling implementation 
and processing stage to ensure that the errors of coding, interpreting, transcription, 
editing and the like are within acceptable statistical limits, hence under the control 


of the auditor. 


Data analysis of audited items, procedures or systems of interest occurs during this 
stage of audit execution; it can range from simple descriptive, non-statistical, 
summaries of collected data, to sophisticated statistical analyses that rigorously 
test predetermined hypotheses related to the operation of the audit entity (e.g. 
correlation and regression analysis; discriminant or logistic regression models; 


categorical analysis models; etc.). 


With respect to the audit process, sampling implementation impacts the process 


in three areas: 


Review: in assisting the limited testing procedures, to verify the accuracy of 
the data collected - this activity typically involves a brief walk-through of 
the system to verify various aspects of the accuracy of the system or data 
under study - information quite useful to the formalization and refinement of 


the plan for sampling implementation. 


Evaluation: in assisting the evaluation of the existing management control 
framework against the predetermined model - which is typically supported by 


evidence gathered by various means for specific evaluative purposes. 


Verification: in determining the principal characteristics of the audit 
populations to be tested, the best source for selecting a sample (and how to 

do it), and the most appropriate method of testing - various aspects of the 
sampling implementation methodology will be required for successful detailed, 


verification testing. 
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The use of sampling implementation principles and practices in auditing is a basic 
prerequisite to conducting sound and reliable audits. Each auditor should be familiar 
with most of the well documented and easy to learn principles and techniques. The 
more advanced methods can be applied with the assistance of experts in statistical 
methods. 


Sample Evaluation 

This stage of the sampling framework (see Figure 8) is mainly concerned with the 
formal analysis of previous audit stages and all collected data that affect the audit 
findings, conclusions and recommendations. 


Sample data can be evaluated from several perspectives, such as: 


- quantitatively: where, for example, population error rates and impacts 


are projected from the sample findings; 


- qualitatively: where, for example, error classification and analysis is 


carried out to explain the particular nature of the errors; 


- for efficiency: amount of useful output gained from the sampling process 


for the effort exerted; 


- for effectiveness: in terms of the validity and reliability of the collection 


instruments used and the objectives achieved; and 


- for risk: expressed in quantitative terms (e.g. sampling error or bias) 
and in qualitative terms (e.g. non-sampling error or bias related to non- 


response, selection problems, processing error, and so on). 


A more detailed checklist of logical and statistical problems with sample data 


analysis, which is usually addressed in this stage by the auditor, follows: 
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¥ Data Problems 
- distributional observations 
(e.g. extreme cases, range problems, small sample sizes, skewed 


distributions); 


- measurement problems 
(e.g. missing data, inapplicable variables, scaling problems, non- 


linearity of measures); 


- sampling problems 
(e.g. weighting of cases, response rates, statistical significance 


and/or confidence levels, bias in estimates). 


® Model Problems 
- failure of data or model to meet required assumptions; 
- transaction or rounding error (e.g., advanced techniques); 
- non-linearity of relationships; 
- non-additivity of relationships; 
- correctness and appropriateness of formulas; 


- computer dependencies or restrictions. 


® Output Problems 
- incomplete or inadequate computer program statistics; 
~ labelling, annotating the report; 
- incompatibility of output with other programs; 
- technical report completeness (e.g. frame specification; sample 
design; statistical analysis and computational procedure; survey, 
sample or frame accuracy; completeness and adequacy; comparisons 


with other sources of information). 


The types of statistical analysis methods that can be applied to any particular 
audit situation are too numerous to cover in this chapter. Currently, there is no 
general, scientific way of carrying out data analysis for all possible applications; 
however, much consensus exists, and is increasing, on data analysis approaches 


and methods for a wide array of particular applications. References found in 
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Appendix VII, 1-20, provide a number of good starting points for the auditor to 
begin or to continue this learning process. 


With respect to the audit process, sample evaluation impacts the following phases: 


Review: in analyzing the preliminary data collected during the limited test 
stage of the audit,so as to develop an initial assessment of the operation of 


management controls. 


Evaluation: in assisting any particular sub-phase of the evaluation stage of 
the audit where data are required to assess specific audit issues,such as "are 
the data supplied by the control system accurate, representative and timely?"; 
in particular, the substantiation of audit hypotheses on the reasons for failure 
to match specified audit criteria can most effectively be addressed with 


sound, reliable sample evaluation data and analysis appropriate to the situation. 


Verification: in conducting the detailed field testing where general or specific 
data and model analysis tools can be efficiently and effectively applied to 
achieve the test objectives. References numbers 17-23 in Appendix VII contain 
illustrations of a number of potentially applicable analysis tools for verification 
testing. 


In the auditing literature, a number of analysis-driven techniques such as analytical 
review and risk assessment, provide particular approaches to sample evaluation 


similar in spirit to the context above. 


The use of sample evaluation techniques and methods is an essential part of every 
auditing operation. Auditors should be familiar with the wide range of such tools 
which may be used to assist them in various phases of the audit process. Good 
technique coverage and documentation exists for the most common applications 

(e.g. descriptive statistics; hypothesis testing techniques; statistical tabular 
comparisons; sampling projections; correlation or regression analysis; etc.). The 
more advanced evaluative methods and techniques can be applied with the assistance 
of experts in statistical methods. 
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Evaluation and Control Theory 


The format of this concluding sub-section under Analysis Techniques differs slightly 
from the above. Because of the wide range and substance of evaluation and control 
models in the literature, only a few highlights of those selected models (see Table 2) 
will be noted, along with the pertinent references. The sub-section will then conclude 
by briefly relating this collection of evaluation and control models to the audit 


process. 
Strategic Planning Models 


This is a suggested extrapolation of the use of well known techniques, usually 
associated with the strategic planning process to internal auditing. (See Appendix VIII, 


A for references.) 


Environmental Sensing Models 


These models involve the systematic application of environmental scanning methods 
for purposes of strategic and operational planning. In the audit context, they are 
applicable to development of strategic and operational audit plans, and to a lesser 
degree to assignment planning, particularly for the identification of relevant 
environmental influences on the nature and operation of the managerial control 


framework. 


Typically, scanning or sensing covers social, political, regulatory, economic and 


technological conditions looking several years into the future. 


Various forecasting technologies are used in the model development process, e.g. 
Delphi, match modelling, trend extrapolation, probabilistic system dynamics and 


cross impact analysis. (See Appendix VIII, B for references.) 
Systems Logic Models 
These models represent the hierarchy of systems and their relations in the entity 


under study. They provide a context for both understanding and guidance to the 


collection of further information, and they assist the auditor in addressing the 
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question "what actions should be performed to produce desired results or to operate 


desired processes in an authorized, economic, efficient and effective manner?" 


Various audit methodologies use the system logic mode! approach; for example, 
auditors in documenting the existing auditee system (see Appendix VIII, C for 
references), or in the development and use.of predetermined control models as 


previously described, use a type of system logic model. 


The explanatory power of a system logic model far exceeds its capability for 
"proving" assertions about an organization; however, weakness in proof capability 
should not be taken as a weakness in the system logic model - other analytic 


techniques are generally needed to complement the use of the system logic model. 
Behavioural Models 


The application of these models, although desirable for rounding out the auditor's 
portfolio of skills, requires considerable additional training. (See Appendix VIII, D 


for references.) 
Quality-control Models 


These models represent the quality control perspective of an organization, system, 
program or activity in terms of the management systems in place and how 
effectively they are, or should be, performing; for example, quality assurance review 
techniques are available (see Appendix VIII, E.1 for reference) which continuously 
monitor the quality status of an audit object, and hence can be used where the 


auditor wishes to implement this approach. 


Various quality control methodologies currently used in the literature, both auditing 
and non-auditing, could be directly adapted to assist the auditor at various stages 
of the audit process. An example can be found in "Auditing for Quality Control" 
(Appendix VIII, E.2) which illustrates a simple scheme for auditing a specific control 


task from a perspective of quality control. 


See Appendix VIII, E.3-E.6 for additional source material related to this perspective. 
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Information Processing Models 


These models are concerned primarily with the human systems of an organization 


from the perspective of information transfer or production agents. 


These models are typically based on behavioral decision-making research, also 
known as human information processing research, and represent a large class of 
powerful frameworks that the auditor might apply directly to the audit context; for 
example, a successful adaptation of this type of model to the audit situation would 
provide a systematic methodology for aggregating control system judgments into a 
meaningful statement on how the system functions as a whole. (See Appendix VIII, 


F for references.) 
Cost-effectiveness Analysis 


This type of analysis refers to studies of the relationships between project costs 


and outcomes, usually expressed as costs per unit of outcome achieved. 


Cost-effectiveness analysis requires monetizing only program costs; benefits are 
expressed in outcome units. This allows comparison and ranking of choices among 
potential programs (audit objects) according to the magnitudes of their effects 
relative to their cost without the necessity of expressing both costs and benefits in 
dollars. Cost-effectiveness and cost benefit calculations for programs whose impacts 


are entirely unknown and non-estimable are not possible. 


Cost-effectiveness is viewed as an extension of cost benefit analysis in assignments 
or projects with multiple and noncommensurable goals. It is based on the same 
principles and uses the same methods as cost-benefit analysis. The assumptions of 
the method, as well as the procedures required for measuring costs and discounting, 
for example, are the same for either approach; therefore, the concepts and method- 
ology noted previously for cost benefit analysis (grouped under Economic and 
Financial Analysis) can also be regarded as a basis for understanding and applying 


the cost-effectiveness approach. (See Appendix VIII, G for references.) 
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Cause-effect Analysis 


This type of analysis for auditing focuses on substantiating hypotheses related to 
the failure to match the specified audit criteria, as part of the evaluation phase of 
the audit process. 


Systematic approaches for carrying out this analysis are given in the references 
found at Appendix VIII, H.1-H.3, each of which was tailored to the particular type 


of audit objective and approach. 


Some form of structured cause and effect analysis noted above is highly 
recommended for each audit application; see the references in Appendix VIII, H.4- 


H.6 for an overview of its use in the development of audit findings. 


Relationship to the Audit Process in General 


With respect to the audit process, the potential scope and use of the evaluation 
control models highlighted in this sub-section are wide ranging and significant. The 
impact of this class of models on selected phases and sub-phases of the audit 
process can be seen in Appendix 1. Coverage of all major sub-phase activity in 
Appendix | is exhaustive (100 per cent) over the range of models suggested above. 
Further details on the specific applications of each model or technique are referred 
to in the noted references in the Appendices, or in Volume III of this Handbook 
(Audit Guides). 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 4, Conclusion - 364 - 


CONCLUSION 


This chapter has presented a compendium of analytic concepts and techniques from 
a wide range of diverse fields and disciplines that have much potential! application 
for the internal auditor. Brief descriptions were given of the actual and potential 
interface between the technique and the audit process, as depicted by its major 


phases and sub-phases. 


The internal auditor is encouraged to further develop and refine such models in the 


pursuit of more efficient and effective decision-making in the audit process. 


The list of references provided in the Appendices falls into two general classes: 
those that can be applied by the auditor and those that will, typically, require the 
help of a specialist. 


Given the unlimited scope of the internal auditors activities, it should not be 
surprising that need will arise for expertise in subject matter, and associated 
methods and techniques, that is not available in a typical group of "core" auditors. 
In these cases the employment of specialists is strongly encouraged in order that 
the same professional audit rigour be maintained in these areas as is currently 


normal for financial areas. 
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Appendix I 


SELECTED PHASES IN THE AUDIT PROCESS CROSS-REFERENCED 
BY ANALYSIS TECHNIQUES AND BIBLIOGRAPHIC SOURCE 


GENERAL CLASSES SELECTED PHASES IN THE AUDIT PROCESS 


OF ANALYSIS PLANNING REVIEW EVALUATION VERIFICATION 
TECHNIQUES 


FLOWCHARTS 
NETWORKS, 
DESCRIPTIVE 
MODELLING 


OAmMoaAweS> 


DECISION 
THEORY 


ECONOMIC 
FINANCIAL 
ANALYSIS 


SAMPLING 
THEORY 


EVALUATION/ 
CONTROL 
THEORY 


x Km am K KKK 


Legend: X : denotes applicable analytic technique by sub-phase of the audit process. 
The sub-phase groupings were derived from the report: Guide to the 
Development and Conduct of Audit Assignments. Exposure Draft, July 
(1982). 


presentations, working papers and similar communication requirements: 


c : denotes sub-phases of the audit process mainly concerned with reports, 
i.e. where the role of analytic methods is minimal. 
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Appendix II 


REFERENCE LISTINGS 


Flowcharts, Networks, Descriptive Models 


A. Documents Analysis 


ie "The Internal Audit Assignment Process", Volume II, Part 1, Internal 
Audit Handbook, 1986. 


2 "Audit Evidence", Volume Il, Part 2, Chapter 5, Internal Audit Handbook, 
1986. 


Be Descriptive Organization Models 


ie "The Internal Audit Assignment Process", Volume II, Part 1, Internal 
Audit Handbook, 1986. 


be "Control: Concepts and Applications for Internal Auditors", Volume II, 
Part 1, Chapter 2; Internal Audit Handbook, 1986. 


3. "A Structured Methodology for the Conduct of Comprehensive Auditing", 


Audit Services Bureau, Supply and Services Canada, 1982. 
C. Process Models 


l. "Guide to the Development, Use and Maintenance of Internal Audit 
Guides", Volume III, Guide 101, Internal Audit Handbook, 1985. 


2. "Flowcharting - Manual Aspects of Accounting Systems", Internal 
Financial Audit Handbook, draft, Treasury Board of Canada (Comptroller 
General), 1983. 


a Analytical Auditing. An Outline of the Flow Chart Approach to Audits, 
Skinner, R.M. and Anderson, R.J., Pitman Ltd., 1966. 
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4, A Guide for Studying and Evaluating Internal Accounting Controls, 
Arthur Andersen & Co., 1978. 


D. Network Models 


L Essentials of Management Science/Operations Research, Buffa, E.S. and 
Dyer, I.S., John Wiley & Sons, 1978. 


E. Predetermined Control Models 


I, "Control: Concepts and Applications for Internal Auditors", Volume II, 
Part 1, Chapter 2, Internal Audit Handbook,: 1986. 


2: "Management Control: Concepts and Practices", Volume II, Part 2, 
Chapter 3, Internal Audit Handbook, 1986. 


3. "A Structured Methodology for the Conduct of Comprehensive Auditing", 
Audit Services Bureau, Supply and Services Canada, 1982. 


F. Matching 


15 A Guide for Studying and Evaluating Internal Accounting Controls, 
Arthur Andersen & Co., 1978. 


Z Auditing: Integrated Concepts and Procedures, Taylor, D.H. and 
Glezen, G.W., 2nd edition, John Wiley & Sons, 1982. 


G. Precedence Analysis 


. Auditing: Integrated Concepts and Procedures, Taylor, D.H. and 
Glezen, G.W., 2nd edition, John Wiley & Sons, 1982. 
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Appendix III 


REFERENCE LISTINGS 


Decision Theor 


AS Issue or Problem Analysis Models 


is "The Question is More Important than the Answer", Stainton, R.S., 


Business Quarterly, 1983, p. 26-31. 


ae "How to Analyze that Problem", Stryker, P., Harvard Business Review, 
July/August 1965, p. 99-110. 


a. "Beyond Implementation: An Analysis of the Resistance to Policy 
Analysis", Mintzberg, H., Infor, Vol. 18, No. 2, May 1980, p. 100-138. 


4, "The Art and Science of Mess Management", Ackoff, R.L., Interfaces, 
Vol. 11, No. 1, February 1981, p. 20-26. 


as "Images of Reality: The Relation Between the Real World and the Model 
World in OR", Buzacott, J., Infor, Vol. 20, No. 3, August 1982, p. 264-272. 


6. "Optimization + Objectivity = Opt Out", Ackoff, R.L., European Journal 
of Operational Research, 1977, p. 1-7. 


B. Mathematical Models 


ie "Mathematical Modelling for Management", Eilon, S., Interfaces, Vol. 4, 
No. 2, February 1974, p. 32-38. 


ri Managing Public Systems: Analytic Techniques for Public Administration, 
White, M.J., Clayton, R., Myrtle, R., Siegel, G. and Rose, A., Duxbury 


Press, 1980. 
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Gs Sensitivity Analysis 


1. "A Scoring Methodology for Assessing the Suitability of Management 
Science Models", Souder, W.E., Management Science, Vol. 18, No. 10, 
June 1972, p. 526-543. 
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CHAPTER FIVE 
AUDIT EVIDENCE 
INTRODUCTION 


Review of audit literature indicates the centrality of the concept of evidence to 

the audit process. The popular text, Management Auditing - Concepts and Practice 
by John A. Edds, states that the ". . . selection, examination, and evaluation of 
evidence is the backbone of any audit because the auditor must amass the evidence 
necessary... to support the points made in the audit report." R.K. Mautz and 

H.A. Sharaf in their much referred to audit text, The Philosophy of Auditing, support 
Edds' view as they state: "Auditing in its entirety is made up of two functions, 

both closely concerned with evidence. The first is the evidence-gathering function; 
the second is that of evidence evaluation." The importance attached to the subject 
of audit evidence by audit researchers is consistent with the views of audit practi- 
tioners. The Standards for Internal Audit which govern internal audit practices in 
the federal government state that "sufficient, valid and relevant evidence should 


be obtained and documented to support the content of audit reports." 


While the concept of evidence is widely recognized by auditors as a key aspect of 
the proper conduct of audit assignments, in practice, the degree of discipline brought 
to evidence-gathering and evaluation duties varies considerably between auditors. 
The variations in practice as to what constitutes sufficient, valid and relevant 
evidence are often explained on the basis of differences naturally arising in the 
exercise of the internal auditor's professional judgment. Although auditor judg- 
ment will always represent a vital aspect of any audit, care must be taken to ensure 
that undue subjectivity is not substituted in areas where objective guidelines could 


be developed and constructively used by the internal audit community. 


ie Edds, John A., Management Auditing - Concepts and Practice, Dubuque, 
Iowa and Toronto, Ontario: Kendall Hunt Publishing Company, 1980, p. 133. 
Zs Mautz, R.K. and Sharaf, H.A., The Philosophy of Auditing, Evanston: 


American Accounting Association, 1961, p. 68. 


3. Treasury Board of Canada, Standards for Internal Audit in the Government 
of Canada, Office of the Comptroller General, 1982, p. 8. 
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The auditing standard of the Certified General Accountants Association (Canada) 
entitled "Auditor Judgement" indicates that there is an increasing pressure on 
auditors to be able to explain and defend their judgments to users of the audit 
report. Given that one of the key problems in auditor judgment is the determination 
of the nature and amount of evidence that should be accumulated, the need for 


objective criteria for such judgments is becoming increasingly significant. 


This document seeks to aid the auditor in determining what constitutes sufficient, valid 
and relevant evidence and how it should be gathered. In this regard, Section One briefly 
explores the position of evidence in relation to other internal audit concepts to 

clearly establish the nature of evidence and the purposes it is meant to serve. 

Section Two discusses the practical aspects of gathering audit evidence, reviewing 

the various types of evidence available and the techniques that can be used to 
accumulate it. Section Three focuses on what is meant by "sufficient, valid and 
relevant" evidence and the factors the auditor should consider when assessing the 
adequacy of evidence gathered to support the contents of the audit report. Section 
Four identifies certain factors that should be recognized in the proper evaluation 


of evidence. 


As a minimum, it is hoped that this structure will provide internal auditors with a 
clearer understanding of references made in the Standards for Internal Audit on the 
subject of audit evidence. More importantly, however, this document should give 
auditors guidelines that will enhance the degree of objectivity and rigour they bring 


to evidence-gathering and evaluation activities. 
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SECTION ONE: EVIDENCE AND ITS POSITION IN AUDIT THEORY 


An understanding of the concept of audit evidence requires an understanding of its 
relationship to other audit concepts. To study why auditors gather evidence, we 
must know the nature and purpose of the audit process as a whole. A fundamental 
knowledge of the audit process provides auditors with the basis on which they can 
develop answers to such problems as: what constitutes evidence, how much 


evidence is enough and how should it be gathered? 


This chapter, then, begins with a brief review of evidence in relation to an overall 
internal audit theory framework. This framework is derived from an external audit 
model modified to reflect the unique features of the internal audit process. A 
detailed comparison of the two audit theory models is provided in Appendix A. 
(Note: in this chapter, the term "external audit" refers to private-sector attest 


audits performed by public accountants.) 
The Internal Audit Model 


In general, internal auditing in the federal government is a function whose purpose 
is to arrive at certain conclusions concerning the condition * of the resources, 
processes and results of the entity under review. Specifically, the Standards for 


Internal Audit expect the auditor to determine whether: 


® systems, procedures and controls are adequately designed and developed, 


and that they are operating efficiently and effectively; 


& adequate information is available for decision-making and accountability 
purposes; 
@ available information is properly utilized in the decision-making process; 
e public funds and assets are adequately protected; and 
nN, This term, as used in the paper, is often referred to as an "assertion" in texts 


which describe audit theory. 
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© legislative, central agency and departmental directives are being 


complied with.” 


Internal auditors usually do not attempt to assess directly the operations and results 
of the entity under review to determine whether the above conditions exist; rather 
the auditors evaluate the adequacy of the control framework established by manage- 
ment to achieve these conditions. Essentially, the control framework serves as a 
proxy measure for determining whether the desired conditions actually exist. Where 
the auditor determines that the control framework is effective, the underlying 
inference is that the entity under review likely achieves satisfactory operating 
conditions. For example, consider the auditor's task of concluding whether the 

level of efficiency found in a system under review is adequate. The auditor's tests 
of the control framework will not give a full appreciation of the system's actual 
efficiency but will give assurance that management is doing everything practical to 
ensure that desired levels of efficiency are achieved. Using tests of the control 
system, the auditor can conclude that there is reasonable assurance that the 


operations are achieving an adequate level of efficiency. 


Since the adequacy of the control framework must be evaluated, a prerequisite to 
the conduct of the audit is the existence of a set of established criteria to be used 
as a basis for the auditor's evaluation. A model controlj framework is developed to 
represent the set of criteria for determining the adequacy of the actual control 
framework in place. As a minimum this model identifies the various objectives 
that should be served by the control framework and the ways in which the auditor 


can judge whether these objectives have been and will continue to be met. 


Evidence in auditing refers to any relevant, reliable matter or facts obtainable 
which will assist the auditor in forming conclusions on the extent to which the 
above-mentioned criteria are met. Facts are obtained by the auditor primarily on 
the actual design, operation and effects of the control system of the entity subject 
to audit, including the actions and representations of those responsible for the 


system's operation. 


a Treasury Board of Canada, op. cit., pp. 61-62. 
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The Standards for Internal Audit generally require that "sufficient, valid and relevant 
evidence should be obtained and documented to support the content of audit reports". 
The Standards go on to identify two factors which should help auditors assess 
whether they have gathered an adequate level of evidence. Specifically, auditors 

are given further guidance relating to (a) the degree of correspondence they must 
seek when comparing evidence to the criteria established for the audit; and (b) the 
degree of proof or evidence required to support the auditor's cenclusions. While 
Section Three of this document explores in detail these factors which influence 
decisions relating to the proper gathering of evidence, it is instructive in this 


overview to discuss briefly the two areas of guidance noted above. 


The extent of evidence required in an audit is influenced by the fact that the auditor 
is not required to determine whether there is an exact correspondence between the 
actual control framework and the control model established for the audit.° Similar 
to the external auditor's concern for "fairness" as opposed to mechanical exactitude 
in the presentation of financial results, the internal auditor seeks "adequacy" in the 
actual control framework, not perfection. The threshold of what constitutes a 
significant deficiency in the adequacy of control performance gives rise to the 

need for a concept of materiality for internal auditors similar in nature to the 


concept used by external auditors. 


In terms of the degree of proof required in an audit, the Standards for Internal 
Audit require that the auditor conduct '". . . examinations and verifications to a 
reasonable extent... .", but do not require"... detailed tests of all transactions".’ 
This statement reduces the auditor's responsibility to gather evidence from one 
where absolute or certain proof must support the auditor's comments to one where 


a reasonable basis for audit comments is acceptable. 


To summarize this analysis of internal audit theory, the obtaining and evaluating of 
evidence is the essence of auditing. The types of evidence obtained depend on the 


nature of the criteria (and ultimately, the underlying conditions) relevant to a 


————— 
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particular audit entity. In addition, the proper evaluation of evidence requires the 
development of explicit criteria which serve as standards for judging the adequacy 

of the controls under review. Finally, auditors are not required to gather all evidence 
that is available to them but only an amount which provides a "reasonable basis" 


for reporting on the "adequacy" of systems, procedures and controls. 


Elaboration on the Role of "Users" and "Established Criteria" 
in the Internal Audit Model 


Users of Internal Audit Reports and their Influence on Audit Evidence 


In this document, the term "user" refers to the primary intended recipient of the 
audit report and the person for whom the auditor performs the audit service. The 
user is to be distinguished from the "auditee" who represents the manager responsible 


for the operations and activities which are the subject of the auditor's examination. 


In general, the primary report recipient can be readily identified. In the federal 
government, the internal auditor performs audits primarily for a deputy minister or 
a department's senior officials. Secondary users of internal audit reports would 
include departmental managers subordinate to those noted above and external audit 
or central agency review groups who may rely upon the work performed by the 


internal auditor. 


The fact that the internal auditor reports to specific "users" is significant in that it 
provides the auditor with an opportunity to receive direction concerning the areas 
where particular audit attention should be focused. This user direction can help 
the auditor in decisions relating to: the relative emphasis to be placed on the various 
conditions that the auditor will conclude upon; the criteria for evaluating the entity 
under review; what constitutes a material deficiency; and ultimately, the nature 
and extent of evidence that should be accumulated by the auditor. The user can 
often provide the auditor with valuable information relating to the types of risk 
which affect an auditee's operations. In the planning of audit coverage, therefore, 
the auditor is well advised to consider this source of information and determine 
how evidence-gathering strategies should be adjusted. We will return to this point 


in Section Three. 
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The opportunity for the internal auditor to receive direction from users is not found 
in all types of audit engagements. For example, in external private-sector auditing 
such user direction is essentially non-existent; it is through the external auditor's 
professional associations that guidelines and standards are developed to aid in deter- 
mining answers to problems relating to such things as the types of criteria necessary 


and extent of evidence required for assignments. 


In summary, internal auditors should ensure that users are approached to offer 


advice concerning: 


@ the relative emphasis to be placed on the various auditee conditions 


subject to audit; 


® the nature of the criteria to be used to evaluate the entity under review; 
and 
e what constitutes a material deficiency. 


This information should influence the auditor's judgment when deciding upon the 


nature and extent of evidence required on any particular assignment. 


The Nature of Established Criteria for Internal Audits and its Effect on Evidence- 


gathering Activities 


As noted, the internal auditor forms various conclusions concerning the degree of 
correspondence between the evidence gathered on the auditee and the established 
criteria. In external auditing, the established criteria are widely accepted by audit 
practitioners and codified by the Canadian Institute of Chartered Accountants in 
the form of generally accepted accounting principles. In internal auditing, criteria 
used to judge the adequacy of controls have not been codified. Toa large extent, 


these criteria, at present, are chosen on a judgmental basis by the internal auditor. 


While auditor judgment is necessary in the determination of criteria, internal 
auditors should not unilaterally establish the criteria that are to be used on any 


given assignment. Instead, auditors should seek input from users and auditees before 
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determining criteria for the audit assignment and obtain further support through 
reference to authoritative sources (e.g. legislation, central agency and department 


policies) where appropriate and feasible. 


Given this perspective on the nature of internal audit criteria, the significant aspect 
to note is the degree of latitude allowed in their determination. In external auditing, 
criteria are relatively fixed while in internal auditing considerable flexibility may 

be allowed. Indeed, it may be that internal audit criteria will never reach the status 
of "generally accepted" but perhaps will require on-going determination on a 


situational basis. 


The nature of the established criteria in the internal audit model gives rise to the 


following effects: 


* The internal auditor should be prepared to change established criteria if 
evidence gathered reveals that the basis for the audit evaluation is 
inappropriate. It is unlikely that external audit criteria would change during 
an audit in that financial reporting is conducted within fairly narrow boundaries 
of acceptable practice. It is quite plausible, however, that the internal 
auditor's criteria could be found deficient as the audit progresses and evidence 


is reviewed. 


e The internal auditor should carefully consider the skills required to conduct 
the audit given the nature of the criteria developed for the assignment. In 
external audits, the nature of established criteria relates to the subject matter 
of "accounting principles". This area is one in which the auditor has been 
specifically trained. In certain cases in the internal audit context, the nature 
of the established criteria may require specialized skills for recognizing appro- 


priate evidence and related techniques for its accumulation. 
Summary 
Evidence refers to facts obtained by auditors which assist them in forming 


conclusions about the adequacy of the auditee's control framework and ultimately 


about the adequacy of the underlying condition of the resources, processes and 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 5, Section | - 391 - 


results of the entity under review. The recognition of the types of evidence 
relevant to a particular audit depends upon the auditor's objectives and the nature 
of the criteria established for the audited entity. Proper accumulation and use of 
evidence therefore depends upon an appropriate understanding of the ultimate 
conclusions an auditor wishes to make and the auditor's use of satisfactory 


evaluative criteria. 


To ensure that audit reports remain relevant to users, auditors should recognize the 
importance of seeking user guidance on the relative significance of various auditee 
conditions that should be concluded upon and the types of criteria they consider 
appropriate to particular audits. Such user influence will affect the relative 


importance of various types of evidence and the amount collected. 


Auditors do not seek exact correspondence between auditee performance and 
established criteria. In addition, auditors are not required to ensure that their 
conclusions are supported by absolute proof; rather a reasonable basis is acceptable. 
These guidelines to internal audit practice reduce the amount of evidence that 
must be accumulated but create problems for auditors in terms of establishing the 
lower limit of audit evidence that will satisfactorily support the contents of their 
report. The problem of establishing the minimum level of evidence required to 
provide convincing audit report conclusions and recommendations will be further 


explored in Section Three. 
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SECTION TWO: TYPES OF EVIDENCE AND METHODS BY WHICH IT IS GATHERED 


To perform properly the evidence-gathering and evaluation functions, an auditor 


must understand: 


« the various types of evidence, how they differ in kind and reliability and 


the various strategies that can be employed to gather them; 


¢ the factors which help determine what constitutes sufficient, valid and 
relevant evidence necessary to support the contents of the auditor's 


report; and 


% the personal qualities that an auditor should maintain and develop to 
ensure that observational errors and bias do not jeopardize the evidence- 


gathering process. 


In this section, the discussion focuses on the types of evidence available to auditors 
followed by a review of the various evidence-gathering techniques that auditors 
can employ. An understanding of the full range of the types of evidence available 
and techniques for its extraction will help give the auditor flexibility in developing 
evidence-gathering strategies. Such knowledge will also help reduce the risk of 
undue auditor reliance on any particular kind of evidence or evidence-gathering 
technique. Section Three will then turn to the factors which help determine what 
constitutes sufficient, valid and relevant evidence. Section Four will review the 
personal qualities auditors should strive to maintain to ensure that evidence is 


properly evaluated. 
Types of Evidence 


In brief, evidence is anything that contributes to the establishment of proof. In 
auditing, evidence includes those things which help the auditor form an opinion or 
belief about the actual conditions which exist in the area under review. In the 
following, evidence is classified from three different perspectives: by form, by 


source and by type of proof provided. 


< 
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Classification of Evidence by Form 


The following list identifies six types of audit evidence according to the criterion 


"what things give proof": 


- Physical presence of objects 

- Observed actions of auditee personnel 

- Oral or written statements and representations 
- Information contained in documents 

- Evidence from reperformance 


~ Mutual consistency between various pieces of data. 
Physical presence of objects 


The physical presence of an object provides proof used to verify the existence of an 


object, assess its qualities or measure the quantity of it on hand. 
Observed actions of auditee personnel 


Observed actions represent proof generated by the auditor. This type of evidence 
is typically used to verify the performance of an activity or procedure and whether 
it is being performed as intended. Observed actions provide important overall 
knowledge of auditee systems and controls and can help the auditor identify 


suspicious circumstances. 
Oral or written statements and representations 


Also called testimonial evidence, this form of evidence represents information 
received from others acting as expert witnesses on matters relating to inquiries 
made by the auditors. Substantial amounts of testimonial evidence are gathered in 
internal auditing from personal interviews or requests for written statements and 
letters of confirmation. This type of evidence can be used for describing most 


aspects of the actual operations under review. 
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Information contained in documents 


This form of evidence is proof derived from various written records such as procedural 
manuals, accounting records, contracts and documents of all types. Records are 
examined to verify the occurrence of transactions or events through examination 

of source documents. Records can also provide a description of the intended design 

of the system under review. Finally, recorded results can be analyzed as a means 


of determining the effectiveness of the controls of the operation under review. 


Evidence from reperformance 


The concurrence of the findings of reperformed actions or calculations with original 
findings is typically used to verify the accuracy of measurements or valuations. 
Checking prices, extensions or other computations are examples of procedures used 
to generate this type of proof. Reperformance of the additions of lists also helps 
check the existence and completeness of listed items, since out-of-balance conditions 
could indicate accidental inclusions, double counting or omissions. Reperformance 
can demonstrate whether the original performance of a control was effective. An 
example would be the repetition of a payroll calculation which a second employee 


had initialled for checking. 


Mutual consistency between various pieces of data 


It can be argued that mutual consistency between various pieces of evidence does 

not in itself represent a new form of evidence but has merely a confirmatory effect. 
Increments of audit evidence pointing toward the same conclusion have a confirmatory 
effect and thus a joint degree of persuasiveness higher than that which any individual 
increment possesses in isolation. The reason, however, for treating mutual consistency 
(or inconsistency) as a type of evidence in its own right is that much important 

audit work is devoted exclusively to searching for it. Analytical review procedures, 
for example, represent systematic study and comparison of related figures, trends 


and ratios in order to identify their mutual consistency or inconsistency. 
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Reliability Considerations 


In general, the intrinsic form of evidence will influence its trustworthiness in 
supporting audit report conclusions. Although one must determine first the 
reliability of the source of various pieces of evidence, secondary consideration of 
the form (physical vs. documentary vs. oral) will give additional means for assessing 
the trustworthiness of the proof provided. Reliability determination is further 


discussed in the following and in Section Three. 
Classification of Evidence by Source 


Audit evidence can be classified according to the source from which it is derived, 
providing the auditor with an additional perspective for looking at the nature of 
evidence. This should enhance the auditor's understanding of what evidence is 
available and thereby increase the auditor's flexibility when determining evidence- 
gathering strategies. Reference to the source of particular pieces of evidence is 


also a useful way auditors can assess the reliability of proof obtained. 
There are four major sources of evidence which will be considered: 


- personal knowledge of the auditor 
- external evidence 
- internal evidence 


- overlapping evidence 


Personal Knowledge of the Auditor 


Direct personal knowledge is normally derived by the auditor through physical 
examination and observation of activities. This type of evidence tends to be the 
most reliable, provided that the auditor can minimize the risk of observational 


errors (refer to Section Four of this chapter). 
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External evidence 


This represents evidence obtained from third parties who are organizationally 
independent from the auditee. There are, of course, degrees of organizational 
independence such as the varying independence of third parties who are within the 


same departments as the auditee and those who are not. 


The reliability of this type of evidence depends on the auditor's evaluation of its 
trustworthiness, competence and objectivity. Where these factors do not prove 
problematic, external evidence generally is favoured as being more persuasive than 


evidence created within the auditee organization. 


Internal evidence 


This represents evidence created by the auditee organization. On any audit, this is 
by far the most prevalent and economical type of evidence to obtain. It also tends 
to be the least reliable (with oral evidence being less reliable than documentary) of 


the various sources of evidence. 


Reliability depends largely on the auditor's determination of the competence and 


trustworthiness of the auditee providing the information. 


Overlapping evidence 


This is evidence which is derived from the mutual consistency of different pieces 


of information pertaining to a control assertion. 


Reliability of evidence from this source depends on the degree of consistency found 
between separate pieces of information, the persuasiveness of the consistency and 


the auditor's skill in assessing the relationship between separate pieces of evidence. 


Each of the forms of evidence discussed can be related to their source. Table 1 


illustrates this and provides examples of each forin of evidence. 


Internal Audit Handbook 
Volume II, Part 2 


Chapter 5, Section 2 


Table 1 


- 397 - 


Forms of Evidence Classified by Source® 


Source and Form of Evidence 


A. Direct Personal Knowledge 
ihe Physical evidence 
2. Concurrence of reperformance 
3 Observed actions of auditee 
personnel 
Ba. -External ‘Evidence 
4, Statements and representations 
by third parties 
Ds External documentary evidence 
(if received directly from 
third parties) 
C. ‘Internal Evidence 
6. Accounting records and reports 
hes Internal documentary evidence 
(obtained from within the 
auditee organization) 
8. Statements and representations 
by management and employees 
D. Overlapping Evidence 
as Consistency with other evidence 
8. 


Example of Evidence 


Actual capital assets or materiel 
available for inspection. 


Concurrence of pricing extensions 
done by the auditor with those 
originally recorded on contractual 
agreements or invoices. 


Performance of security routines by 
auditee personnel. 


Personal interview of user group 
personnel as to satisfaction with 
auditee services. 


Studies or reports prepared by subject 
matter specialists. 


FINCON reports. 


Internal management information 
system reports. 


Explanations of auditee procedures. 


Analysis of trends such as customer 
complaints and correlation with other 
evidence such as the known condition 
of the auditee's control system. 


Anderson, Rodney J., The External Audit 1 - Concepts and Techniques, 


Toronto 1977, p. 254. 
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Classification of Evidence by Type of Proof Provided 


Recognition that evidence can provide different types of proof is useful to auditors 
when designing their evidence-gathering strategies. Searching for pieces of evidence 
which provide different types of proof enhance the overall persuasiveness of the 
auditor's conclusions. In this part, type of proof provided is looked at from two 
perspectives. First, does the evidence gathered provide positive or negative proof? 
Second, does the evidence gathered represent primary, corroborating or contradictory 


proof? 


Positive versus negative 


Evidence which provides positive proof is evidence which directly supports a 
proposition being verified. For example, if an auditor is attempting to verify that 
a control is adequate, evidence which generates positive proof provides direct 


assurance that the control is indeed adequate. 


Evidence which provides negative proof is actually the absence of evidence, after a 
reasonable search for it, which contradicts the proposition being verified. For 
example, in verifying the adequacy of a control, the auditor would first determine 
the types of conditions that would likely exist if the control was inadequate. The 
auditor would then look for the existence of such conditions. Where the auditor 
failed to discover conditions suggesting control inadequacy then negative proof has 
been generated. This proof provides some evidence that the control is actually 


operating adequately. 


Evidence which provides positive proof is inherently more reliable than evidence 
generating negative proof. To find that a certain condition exists is more reassuring 
than not finding something wrong. Tests to establish the effectiveness of controls 
however, often employ verification procedures involving a search for negative 
proof. To search for undesirable effects is sometimes easier than proving that a 


control is indeed operating effectively. 


— 
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Primary, corroborating and contradictory 


Evidence may be classified as to whether it is primary, corroborating or contradictory. 
Primary evidence, as the name suggests, is the evidence upon which the auditor 


places primary reliance in establishing the proof of audit report conclusions. 


Depending upon how persuasive primary evidence is, there may or may not be a 
need for additional evidence. Where additional evidence is gathered it may be 
considered corroborating or contradictory. Corroborating evidence supports the 
type of proof provided by the primary evidence while contradictory evidence refutes 
it. Where additional evidence gathered is contradictory, it will usually require the 
auditor to extend audit testing procedures to confirm or refute the apparent 


contradiction. 


Summary 


Three ways of classifying evidence have been presented. Recognition of the various 
forms, sources and proofs of evidence should enhance the flexibility of the auditor's 
evidence gathering strategies and reduce the risk of undue reliance on any one 


particular type. 


Methods of Gathering Evidence 


The quality of audit results is necessarily dependent on the adequacy of the methods 
used for gathering evidence. In current internal audit practice much reliance is 

being placed on interview inquiry as a means of gathering evidence. In many cases, 
this practice is probably justified because of the limited usefulness of other techniques. 
Nevertheless, internal auditors must be wary of habitual reliance on certain 

techniques and ensure that for each assignment the full spectrum of evidence- 
gathering methods is considered. 


In this part, there is a consideration of the techniques generally available to the 
auditor. These are presented schematically in Table 2 in relation to the various 


sources and forms of audit evidence previously discussed. 
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Audit Evidence and Audit Techniques” 


Source of 
Audit Evidence 


A. Direct Personal 
Knowledge 


B. External Evidence 


cS Internal Evidence 


D. Overlapping Evidence 


9.  Ibid., p. 255. 
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Table 2 


Form of 
Audit Evidence 


Physical evidence 


Concurrence of 
reperformance 


Observed actions 
of auditee personnel 


Statements and 
representations by 
third parties 


External documentary 
evidence (received 
directly from third 
parties) 


Accounting records 
and reports 


Internal documentary 
evidence 


Statements and 
representations by 
management and 
employees 


Consistency with 
other evidence 


Most Closely Related 
Audit Technique 


L. 


Physical examination 


Reperformance 


Observation 


Inquiry 


Scrutiny, 
vouching, 
analysis 


Scrutiny, vouching, 
analysis 


Scrutiny, vouching, 
analysis 


Inquiry 


Correlation with 
related information 
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Physical Examination 


Physical examination generally involves the measurement of the quantity of an 
asset or assessment of its quality as compared to a predetermined standard. The 
technique can also search for negative proof such as a review for evidence of 


shortages or damage in assets. 


Physical examination is often used to test the effectiveness of controls particularly 
for those controls relating to the security of physical quantities or qualities of 
tangible assets. Evidence is gathered on items having a tangible presence. Materiel, 


for example, is often a subject for this form of audit test. 


The reliability of evidence obtained from this method will depend on whether 


specialized skills are required of the auditor to avoid observational error. 
Observation 
The elements of observation include: 


~ identification of a specific activity to be observed; 
~ observation of its performance; 
- comparison of observed behaviour against criteria; and 


- evaluation and conclusion. 


Normally, observation serves objectives such as the verification of the performance 
of various control procedures. To the extent that the auditor may observe suspicious 
activities in the entity under review, the test may be thought to also test the 


effectiveness of control procedures. 


Usually, observation is important where it can be employed economically. Again, 
the reliability of evidence derived from this method depends on whether specialized 
skills are required of the auditor given the complexity of the activities under 


review. 
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Inquiry 

Oral Inquiry - Auditee 

This type of evidence is by far the most easily obtainable but it tends to be the 
least reliable. In general, all oral auditee representations relating to "material" 
items must be corroborated with other evidence. 

Inquiry can be used in several ways: 


- as a means of determining the nature of the control sytem under review; 


- to obtain explanations of unusual items discovered in the course of 


gathering other evidence; and 

- to elicit information otherwise not available. 
The reliability of oral inquiry depends upon the objectivity and knowledge of the 
employee providing the information. In addition, the competence of the inquirer 
(knowledge, tact, objectivity and judgment) is crucial to the reliability of this 
technique. 
Written Inquiry - Auditee 
Typically, this technique is used in the internal audit context at the report writing 
stage when the auditee is requested to comment in writing on the factual validity 
of audit report representations. 
It is useful to ensure that such evidence is obtained: 


- to act as a final check against auditor interpretation of the facts; 


- to ensure that the auditee has comprehended the auditor's conclusions; 


and 
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- to help emphasize the auditee's responsibility to take action on the 


findings. 


Inquiry - Third Parties 


In this document, third parties include all persons outside of the auditee organization. 


This may include: 


- officers and employees of the same department but outside the auditee 
unit (provided such persons are truly independent of the auditee - see 


below); 


~ officers and employees of other departments; and 


- specialists not included in the above noted groups. 


Evidence from third parties is generally considered more reliable than auditee- 
generated evidence. Reliability, however, is dependent on the trustworthiness, 
competence and independence of the third party and the directness and effectiveness 
of the auditor's communication. Internal auditors must be particularly wary of 
possible competing interests between auditees and third parties as such a situation 


would likely diminish the usefulness of third-party inquiry. 
Inquiry directed towards user groups of auditee services or specialists are often 
quite useful ways of gathering corroborating evidence. In using specialists, however, 


the auditor should: 


= establish the specialist's competence through reference to qualification 


and reputation; 


- ensure that communication is effective to the specialist through the use 


of a clear statement of purpose and requested work; 


- ensure that the specialist states assumptions and methods; and 
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- auditor reviews specialist work once complete and tests it for 


reasonableness. 


Vouching/Scrutiny 


Vouching is the verification of a given action through examination of related source 
documents that provide the needed proof. This technique can be used to compare 
the action under consideration with documented facts and to investigate any 


differences. 


Scrutiny involves a searching review of data to locate significant items requiring 
further investigation. Generally, this technique is used to provide assurance that 
controls are operating effectively as supported by the absence of suspicious circum- 
stances which would suggest possible deficiencies. The reliability of this type of 


evidence is largely dependent on the skill and experience of the observer. 


Analysis 


In this document, analytical review relates to the analysis of the system under 
control so as to obtain evidence as to the effectiveness of the control system or as 


a means of identifying the causes and effects of any control deficiencies. 


Generally, analytical review involves the comparison of normal trends in the entity 
under review with expected results. The relationships arising from this comparison 
must be reasonably explainable, otherwise there is likely a need to gather additional 


evidence to clarify the illogical or unexpected relationships which seem to exist. 
Correlation 
This technique looks for the mutual consistency between different pieces of 


evidence. It is used to provide corroborating evidence which increases the 


persuasiveness of the auditor's proof. 
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Summary 


In Section Two, we reviewed the various types of evidence available to the internal 
auditor and the techniques that can be employed when gathering it. The focus of 
the discussion has been to encourage auditors to consider the full range of evidence 
that may be brought to bear on any audit assignment. It is hoped that the risk of 
undue reliance on certain kinds of evidence or evidence-gathering techniques will 


be reduced as a result of this discussion. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 5, Section 3 - 406 - 


SECTION THREE: SUFFICIENT, VALID AND RELEVANT EVIDENCE 


In this section of the chapter, attention is focused on the factors which help 
determine what constitutes sufficient, valid and relevant evidence necessary to 


support the contents of the auditor's report. 


The Importance of Objectively Determining Sufficient, Valid and Relevant Evidence 


As noted in Section One, the Standards for Internal Audit require that auditors 
",..conduct examinations and verifications to a reasonable extent..." but they are 


10 Because auditors 


not required to perform"... detailed audits of all transactions". 
are not required to gather all evidence available to them, the problem arises as to 
what actually constitutes sufficient, valid and relevant evidence which will adequately 


support the contents of audit reports. 


General rules or criteria by which the auditor can arrive at an assessment as to the 
adequacy of the evidence gathered have not been codified. At present, conclusions 
concerning whether the evidence gathered is sufficient, valid and relevant are 
typically arrived at through the exercise of the auditor's professional judgment. 
While it is recognized that the internal auditor's judgment will always play a central 
role in such decisions, there are several compelling reasons for ensuring that auditors 
attempt to enhance the degree of objectivity brought to evidence assessments 

where possible. 


First, where too much subjectivity is allowed in determining what constitutes 
sufficient, valid and relevant evidence, there is a danger that different auditors 

will choose varying types and amounts of evidence despite the audit being conducted 
under essentially the same circumstances. As a result, the information value of an 
audit report may also be expected to vary given the differing levels of detail upon 


which its contents are based. 


10. Treasury Board of Canada, op. cit., p. 77. 
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A second reason for ensuring objectivity in evidence-gathering decisions is derived 
from the interest users are showing in the internal audit function. As the audit 
function matures in the federal government, more use is being made of audit reports. 
It is not unrealistic to expect auditors to disclose to various users the evidence 
supporting the contents of their report and the basis upon which the auditor 
determined the adequacy of the level of evidence gathered. Groups such as depart- 
mental audit committees, central agency review groups and the Auditor General of 
Canada all may have a need to rely on the internal auditor's work. It is quite proper 
that they establish a basis for this reliance by reviewing the evidence-gathering 
decisions made by the internal auditor. With an objective basis for determining 
such decisions, the internal auditor may provide the rationale from which a wider 


reliance on the internal audit function can take place. 


A final reason for ensuring enhanced objectivity is derived from the existence of 
increasing resource constraints in the federal government. While there is increasing 
use of audit reports, resource constraints require that auditors carefully consider 
what level of effort can practically be applied to the gathering of evidence to 
support audit conclusions. An explicit awareness of the factors associated with the 
trade-off between higher quality audit results and resource limitations may improve 
the auditor's judgment as to what type and level of audit effort will be most 


beneficial given resource constraints. 


To enhance the objectivity of evidence-gathering activities the following section 
amplifies what is meant by sufficient, valid and relevant evidence and the factors 
which should influence an auditor's decision as to when an adequate amount of 


evidence has been gathered. 
Sufficiency, Validity and Relevance of Evidence - Definitions 


The concepts of sufficiency, validity and relevance are interrelated. As used in the 
Standards for Internal Audit, sufficiency is a measure of the quantity of audit 
evidence obtained and validity and relevance are measures of its quality. The 
decision as to whether a sufficient quantity of evidence has been obtained will be 


influenced by its quality. 
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To be sufficient, evidence must be persuasive to an extent which justifies the 
contents of the audit report. Sufficiency is achieved when both the auditor and the 
recipient of the audit report are satisfactorily persuaded that the auditor's findings 
and conclusions are appropriate. This normally requires that the amount of evidence 
gathered is sufficient to also convince the auditee, but ultimately the auditor must 
be most concerned about persuading the user of the audit report who has control 


over the implementation of the audit recommendations. 


Validity refers to the soundness or credibility of evidence in supporting the auditor's 
conclusions concerning the nature of the entity under review. Generally, the more 
reliable the source and form of audit evidence, the more valid it will be. The 
generalizations about the reliability associated with different forms of evidence 
from various sources was discussed in Section Two of this chapter. Table 3 is an 
illustration of a reliability ranking scheme that contains the points made in Section 
Two and that auditors have found useful over time. While it is important to 
recognize that there will always be exceptions to any generalization, this ranking 
scheme should help auditors assess the probable reliability of various pieces of 


evidence. 


Relevance means the degree to which audit evidence relates to the auditor's 
objectives. Audit objectives can be divided into compliance and substantive 
objectives. Compliance objectives are concerned with assessing the adequacy of 

the design and operation of the essential controls of the entity under review. 
Specifically, audit evidence derived from compliance procedures is related to 
verifying: (a) the existence and completeness of essential controls, (b) whether 

they are operating as designed, and (c) whether they are effective. Substantive 
objectives are concerned with substantiating significant deficiencies and major 
inefficiencies in the design and operation of essential controls, to assess their effect 


and to identify their causes. 


Clearly the auditor must seek evidence for each of these individual objectives. 
Obtaining the most persuasive evidence as to the existence of a control! will not 
compensate for failure to establish its effectiveness. Audit evidence must be relevant 


to the specific audit objective it serves and each specific objective must be ultimately 
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Note: 


Table 3 


Reliability of Evidence given its Source! ! 


Diminishing 

Degree of 

Reliabilit 
Direct personal knowledge obtained by the auditor through 


direct observation, physical examination, recomputation, etc. 


Evidence obtained through confirmation with, or inquiry of, 


competent, trustworthy and independent third parties. 


Documentary evidence obtained through the vouching of records 
produced external to the auditee (e.g. specialist reports, 


reviews). 


Documents developed, under satisfactory conditions of internal 


control, by auditee. 


Representations of auditee organization; sub-divide represen- 
tations according to the following categories ranked in terms of 


reliability: 


- representations of trusted senior officials; 

- representations from officials with a relatively more 
objective point of view of specific matters; 

~ representation can be corroborated by representations of 


a number of other officials. 


This chart is based on the assumptions that, in general, direct 
personal knowledge is more reliable than indirect evidence 


and external evidence is more reliable than internal evidence. 


ll. 


Anderson, op. cit., p. 252. 
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covered. It is instructive to consider the extent of audit evidence that has been 
gathered for each audit objective. Because evidence concerning control design and 
operation may be more prevalent or more easily obtained than evidence relating to 
the effectiveness of its operation, there may be a tendency within certain audits 
performed to focus disproportionately upon control processes without due consider- 
ation of the results being achieved by the operation of those controls. Without a 
balanced coverage of each audit objective, however, the overall usefulness of the 


audit is greatly diminished. 


With these meanings for sufficient, valid and relevant evidence in mind, the next 


part indicates how auditors may test for those attributes. 
Testing for Relevance, Validity and Sufficiency 


The collection of data on auditee activities occurs during all phases of the audit. 
Beginning in the planning phase, data relating to the auditee are collected in stages. 
Initially, a limited amount of data is gathered on each area subject to review and 
assessed for relevance, validity and sufficiency given defined audit objectives. 
Where data gathered are insufficient to allow the accomplishment of audit 
objectives, the auditor is generally obliged to gather additional information. The 
gathering of incremental information, however, is only done where the benefits (in 


terms of audit results) exceed the costs associated with its collection. 


The following represent guidelines which can be used to test collected data as to 


whether relevant, valid and sufficient evidence has been obtained. 
Relevance 

Tests 

Determine whether the data gathered relate to: 


- the activities included within the defined scope of the audit; 
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- either a compliance or substantive verification objective (refer to the 


definitions previously provided in this section); 


- a material concern. 


Comments 


Where pieces of data gathered are proven relevant, they then become useful to the 
auditor as evidence. In this context, data relate to any facts or matter available 
to the auditor. Evidence is distinguishable from data in that it provides facts or 
matter which are relevant to audit objectives and which will assist auditors in 


forming their conclusions about the entity under review. 


In addition to considering relevance in absolute terms, the auditor should recognize 
that there exists degrees of relevance depending on the directness of the relationship 
between the evidence and the auditor's objectives. If the relevance of a given 

piece of evidence is extremely low, it is usually more economical to examine some 
alternative, more directly relevant evidence than to spend time gathering and 


evaluating evidence of indirect significance. 


Validity 


Tests 


Determine the soundness, trustworthiness or defensibility of the evidence gathered 


by assessing the reliability of: 


> the source of the evidence gathered (personal knowledge, external, 


internal, overlapping); 


- the intrinsic nature of the evidence (physical evidence vs. documentary 


evidence vs. testimonial evidence, etc.) 


- the method used to gather the evidence (physical examination vs. inquiry 


vs. analysis, etc.) 
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Comment 


As noted in the comments for relevance, the less reliable a given piece of evidence 


is, the more likely it will be uneconomical to collect it. 


Sufficiency 


Tests 


Determine whether the evidence gathered is persuasive to an extent which justifies 
the expression of an audit conclusion relating to the matters subject to the auditor's 


examination. 
Comments 


In Section One, we noted that auditors are not required to obtain absolute proof in 
the support of their conclusions. The test for sufficiency of evidence gathered 
derives from the auditors requirement to conduct "...examinations and verifications 


i2 


to a reasonable extent.""~ A criterion which may help serve the auditor in 


evaluating whether sufficient evidence has been gathered may be stated as: 


- a sufficient amount of evidence has been gathered when it satisfies the 
"degree of confidence" an auditor and the user of the audit report wish 


to have in relation to audit findings and conclusions. 


Where the evidence gathered does not meet sufficiency requirements the auditor is 
faced with the decision as to whether or not additional information should be gathered. 
The auditor must decide whether the cost of additional information gathering is 
warranted given the benefits associated with the particular audit objective subject 


to verification. 


12. Treasury Board of Canada, op. cit., p. 77. 
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Summary 


The above tests have been suggested as a means by which auditors can test pieces 
of data gathered as to whether they constitute sufficient, valid and relevant 
evidence. There are four particularly problematical aspects to the tests noted 
above. First, in tests for relevance, an auditor must have a method for determining 
the degree of relevance and what constitutes a material concern. Second, in tests 
of validity, the auditor must weigh the credibility or reliability of the evidence. 
Third, in tests for sufficiency, the auditor must understand how to assess whether a 
satisfactory "degree of confidence" has been achieved in relation to the audit 
conclusions. Finally, the auditor throughout the evidence-gathering process must 
be aware of the implicit cost/benefit relationship that exists. More evidence 
improves the clarity and persuasiveness of the auditor's conclusions but each 
increment in the data gathered is only obtained at a cost. The balance of Section 


Three will provide guidance to auditors on each of these areas. 


Factors Influencing the Assessment of Sufficient, Valid and Relevant Evidence 


By way of introduction, Figure | illustrates schematically the factors and relation- 
ships covered in detail in the balance of Section Three. It is suggested that the 


reader refer back to this overview as each factor is discussed in turn. 


Materiality 


The concept of materiality in auditing was first derived by external financial attest 
auditors. In such audits, it is unnecessary for the auditor to ensure that absolute 
accuracy is present in the reporting of financial results. Such accuracy is neither 
attainable given the nature of the financial reporting process nor justifiable given 
the needs of users of financial statements. As long as a "reasonable" level of 
accuracy is achieved, the auditor should be free to express the opinion that the 
financial statements are fairly presented. This recognition of a threshold of 
accuracy below which it is unnecessary and undesirable to gather evidence is 


embodied in the concept of materiality. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 5, Section 3 - 414 - 


In the context of external auditing, materiality is defined as "...the limit of 
acceptable error or omission above which distortions in a set of financial 
statements... are sufficient as to destroy the fairness of presentation or misinform 


and misdirect the Peaderi.tse7 


While the concept of materiality is generally accepted in principle by external 

auditors, its objective application to actual audit decisions has been problematic. 
Due to the impracticality of communicating directly with users, external auditors 
exercise their professional judgment to determine the dollar value limit of errors 


which they consider to be acceptable. 


The principle behind materiality is as relevant to the internal auditor as it is to the 
external auditor. In short, the internal auditor does not expect perfection and 
infallibility in the design and operation of policies, procedures and controls but 
seeks a condition of adequacy. The term adequate implies that certain errors 

or deviations in the design and operation of auditee systems can be tolerated if 
they are below a particular threshold level of error size or frequency. The question 
of what constitutes an "adequate" policy, procedure or control then, involves the 


establishment of internal audit materiality guidelines. 


In internal auditing "...an item would be considered material if an error in it (or its 
complete omission) would cause prudent, intelligent information users to change 
decisions that they might otherwise make on the basis of information provided by 
and about the auditee."!4 According to this definition, internal audit concerns go 
beyond an interest in the strictly economic aspects of items under review. Asa 
general rule, a contro! which relates to an item which is of material concern to the 
user of the audit report should be included within the scope of an auditor's 


examination. 


13. Eddsyopseit., p. 144. 


14, Anderson, op. cit., p. 127. 
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FACTORS INFLUENCING THE ASSESSMENT OF SUFFICIENT, 


VALID AND RELEVANT EVIDENCE! 
Materiality Practical 
Required Level Availability of 
of Confidence Audit Evidence 
Time Cost 
Level of Ability to Minimize Constraints 
Required Detail Risk of Error 
Minimize Minimize 
Failure to Error of 
Detect Material Improperly 
Errors or Concluding 
Deficiencies Errors Exist 


Minimize 


Relative 
Risk 


Pressures to Gather Greater Pressures to 
Levels of Evidence Limit Amount 
of Evidence 
Gathered 


Nature and Quantity 
of Evidence Gathered 


Figure | 


ee EEEEEEEEEEEEEEEEEEEemeed 


15. Ibid, p. 127. 
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The practical problem for internal auditors is to determine what errors would 
significantly influence the decision-making process of users should they be disclosed. 
Failure to establish a proper materiality limit will result in the gathering of audit 
evidence and the reporting of audit results at an inappropriate level of detail. A 
threshold limit which is set at too high a level of detail will result in audit report 
comments which are too general to be of significant use to the reader of the audit 
report. Threshold limits which are too low can result in "over-auditing" and the 


cluttering up of audit reports with information at an overly precise level of detail. 


Internal auditors, like external auditors, must use their professional judgment 

when establishing the level of detail covered by their review. Table 4 lists a number 
of factors the auditor may wish to consider when determining which items in the 
system under review constitute material ones and consequently require audit 


evaluation of their related controls. 
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Table 4 


Materiality Guidelines 


The following represent factors the auditor should consider in determining whether 


items under review are material. The auditor, with appropriate input from users, 


should review items in terms of whether they involve: 


known or probable management issues or concerns; 


areas of particular on-going concern identified in previous audits; 


questions of non-compliance with financial regulations, fraud or other 


irregularities; 

potential areas of uneconomical or inefficient operations; 
uncertainty concerning the entity's knowledge of the effectiveness of 
its programs and deficiencies in the entity's procedures to evaluate 


effectiveness; 


a program or activity of particular interest because of its nature or 


relative size, and its importance or impact; 


significant new or expanded programs or activities; 


unusual program management characteristics, such as restrictions or 


freedoms in carrying out functions; and 


financial, human and physical resources of particular interest because 


of their nature and importance. 


It is likely that where an item involves at least one of these factors, it will be 


"material" to the user of the audit report. Evidence-gathering strategies should be 


established so that the controls relating to all material items are evaluated. 
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One can identify an obvious parallel between the factors an auditor considers when 
determining what constitutes a material item and the factors considered when an 
auditor selects audit assignment units in the audit planning process. In both cases 
the auditor's interest is to ensure that audit results focus on items of significance 

to users of the audit report. The only major difference in the auditor's planning 

and materiality decisions is one of scale. A materiality limit represents the dividing 
line between items of significance and insignificance. Audit assignment units, 
though also chosen on the basis of significance to audit report users, involve a 
higher level of abstraction and will include therefore, a number of individually 
material items (refer to Volume I, Chapter 3 for a detailed discussion of the auditor 


program planning process). 


Internal auditors have one particular advantage over external auditors when it 
comes to establishing materiality limits. The users of internal audit reports are 
identifiable persons who can be asked to provide input concerning their views on 


what constitutes a material item. 


We have noted that the audit process involves essentially the collection of evidence 
in order to support the contents of the audit report. In deciding which controls to 
investigate and how much evidence to collect, the auditor must consider the 
"materiality" of the items subject to control. A matter should be judged material 
if knowledge of errors in it would likely influence the user of the audit report to 
alter decisions otherwise made on the basis of information provided by and about 


the auditee. To summarize, an internal auditor should: 


e ask users of audit reports, where practical, to provide input concerning 


what constitutes a material matter; and 


» make explicit on all assignments, statements as to which items constitute 
material matters, which items are immaterial and the basis for such 


judgments. 
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Required Level of Confidence 


Beyond materiality considerations, the amount and type of evidence gathered by 
auditors should also depend on the degree of confidence they wish to have in the 
validity of audit conclusions given the needs of the users of the audit report. In 
auditing, 100 per cent certainty is not possible, and confidence is used to represent 
the degree of certainty an auditor has that an error or omission has not been made 


in arriving at audit report conclusions. 


Absolute certainty in an audit report is not strived for because: 


- audit evidence is rarely conclusive in the first place and therefore 


absolute certainty may be unattainable; and 


- audit testing must be limited to make the activity economically justifiable. 


In this part, the means by which auditors can assess and influence their level of 


confidence about the validity of audit report conclusions is discussed. 

Types of Errors Auditors Can Make 

The question of whether a reasonable level of confidence in the validity of audit 
reports has been obtained, can be examined in terms of the complement of such 


confidence: the risk that the auditor's conclusions are in error. 


Auditors can make two types of errors in the contents of their audit reports. They 


can: 


conclude that no material control deficiencies exist in the entity under 


review when, in fact, such deficiencies exist; or 


e conclude that material control deficiencies exist in the entity under 


review when, in fact, such deficiencies do not exist. 
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Concluding that no material control deficiencies exist when, in fact, they do, isa 
much more likely type of error auditors can make. In general, it is normal practice 
that auditors further substantiate the existence, causes and effects of any material 
deficiency which is detected. Such further exploration of identified weaknesses, 
along with the auditee challenge process inherent in the conduct of an audit, will 
likely minimize the significance of the risk of the auditor wrongly concluding that 
errors exist when, in fact, they don't. As such, the following discussion focuses 
upon the means by which the risk of undetected control deficiencies can be assessed 


by the auditor and minimized. 
Minimizing the Likelihood of Undetected Control Deficiencies 


In general, the internal auditor's method of minimizing the likelihood of undetected 
control deficiencies involves three steps. First, the auditor studies the auditee's 
operations in terms of the existence of conditions that will make effective control 
inherently difficult to achieve. Table 5 identifies a number of factors which are 
potentially significant in assessing control deficiencies. Review of these factors in 
qualitative terms will give the auditor a rough measure of the inherent risk of 


control ineffectiveness. 


As the second step in minimizing undetected control deficiencies, the auditor should 
assess management's actions to reduce the likelihood of control problems. In this 
instance, the auditor is generally interested in management's awareness of the 
factors which could cause control problems and what steps they have taken to ensure 


that such problems do not occur. 


The risk that controls will be ineffective given the inherent nature of the entity 
under review and management's efforts to reduce control deficiencies is termed 
"relative risk" in this document. The significance of relative risk is that its 
assessment by the auditor is crucial as a basis for the third step auditors must take 


to minimize the likelihood of undetected control deficiencies. 
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12. 


13. 


14, 


15. 


Table 5 
Risk Factors 
Known aspects concerning the design and past performance of the control 
system. 


Competence of Management - The less competent the management, the higher 
the risk of control deficiencies. 


Size of Unit - The larger the unit under review, the greater the magnitude of 
potential losses; therefore, the greater the demand for control within narrow 
error limits. 


Recent Changes - Probability of control system deficiencies likely to be 
greater during "break-in" period. 


Complexity of Operations - Probability of error increased. 


Liquidity of Assets - Possible target for defalcations thereby putting control 
system under increased pressure. 


Economic Condition of Unit - The risk of control breakdowns is often greater 
in units which are under significant economic constraints. 


Rapid Growth - Rapid growth stretches the personnel and management control 
system of an operation. 


Extent of Computerized Operations - The degree to which controls are built 
into the processing functions of a computer may reduce visibility and attention 
paid to adequacy of control. 


Time Since Last Audit - Effects of an audit diminish over time. 


Pressure on Management to Meet Objectives - For essentially the same reasons 
noted in item 7. 


Extent of Central Agency or Other Regulation - Higher-level controls may 
reduce overall probability of lower-level control deficiencies. 


Level of Employee Morale - Low morale may be indicative of high control 
risk situations. 


Political Exposure. 


Work of External Auditors. 
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The determination of relative risk gives the auditor an initial indication of the 
likelihood of control deficiencies. Where the auditor believes that the probability 

of material deficiencies is high, the auditor will want to ensure that the audit 

tests performed and evidence gathered are more extensive than the situation where 
relative risk of control deficiencies is low. Increasing the extent of testing improves 
the auditor's confidence as to the likelihood that all material control deficiencies 


have been detected. 


In summary, for two audit situations which differ in terms of relative risk the 
internal auditor is obligated to gather more evidence in that situation where relative 
risk is higher, given that the same degree of confidence is to be associated with the 


auditor's conclusions. 
Audit Risk 


Audit risk is the chance that material control deficiencies exist and are not detected 
by the auditor during the audit. Essentially an internal auditor's confidence in the 
validity of audit report conclusions is derived from the assessment of the chance of 


error indicated by audit risk. 


The risk that audit procedures fail to detect material control deficiencies may 


arise from these different areas: 


- an improper assessment of relative risk which resulted in a faulty design 


and scope of audit procedures (a specific form of non-sampling error); 


- the possible impropriety of the auditor evaluation process as a result of 
faulty assumptions, observational bias and illogical conclusions (general 


forms of non-sampling errors); 


- the inherent statistical problems of using a sample to make general 
conclusions about a system as a whole (the risk that the sample chosen 
is not representative of the larger population - a form of sampling error); 


and 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 5, Section 3 - 423 - 


- the inherent limitations in the effectiveness of chosen audit procedures 
(regardless of how well performed, an audit procedure may be unable to 
detect a certain kind of deficiency because it is the wrong procedure or 


because the underlying system is not susceptible to verification). 


These sources of audit risk can never be eliminated but a specific awareness of 
them allows auditors to reconsider the measures they have taken to minimize the 


existence of error in their audit report comments. 


Statistical sampling provides the auditor with a quantified measure of sampling 
error. Control over other forms of audit risk, however, is largely dependent on the 
competence of the audit staff who must ensure that the audit is performed according 


to the highest standards of due care. 
Auditor Confidence - Summary 


An auditor may best assess whether a reasonable level of confidence in the validity 
of audit conclusions has been obtained through analysis of the complement of such 


confidence: the risk that the auditor's conclusions are in error. 


The preceding discussion on how the likelihood of auditor error can be assessed is 
illustrated in Figure 2. This figure shows that both management's efforts to ensure 
that effective controls exist, and the procedures carried out by the internal auditor, 
are screens which prevent the occurrence of undetected material control deficiencies. 
The auditor's confidence is ultimately based on the auditor's satisfaction that audit 
evidence-gathering strategies have been adequately adjusted to detect material 
control deficiencies. Adequate adjustment of the audit evidence-gathering strategies 


is further dependent on the adequacy of the auditor's assessment of relative risk. 


Practical Availability of Audit Evidence 


In the pursuit of being reasonably confident that no material errors exist, the 
auditor must necessarily be influenced by the practical constraints of time and cost 


on the quantity and quality of evidence available for review. 
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ASSESSMENT OF AUDIT RISK 


Auditee Auditor’s efforts Undetected 
actions to to detect control control 
ensure effective deficiencies deficiencies 
controls 


Inherent 
potential 
problems 

in effectiveness 
of controls 


Factors which 
give rise to 
control 
deficiencies 
(See Table 5) 


Relative Risk: 

— the chance that 
control will be 
ineffective in the 
first place because 

of the inherent nature 
of the entity under 
review, mitigated by 
management’s efforts 
to ensure effective 
control. 


Audit Detection Risk: 


— the chance that 
controls will have 
material deficiencies 
and they are not 
detected by the auditor 
during the audit. 


Figure 2 
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All audits face time and cost constraints. Audit reports which are delayed too long 
will be useless to the user. The cost of gathering audit evidence must not outweigh 
the utility derived from the audit report. Thus, although more precise and reliable 
audit evidence may be available to the auditor, consideration must be given to the 
additional cost or extended timeframe that must be incurred to gather the additional 


evidence. 


The study of cost/benefit relationships is typically not formally done by auditors. 
It is clear, however, that such a relationship exists in audits. While readers of 
audit reports benefit from increased auditor confidence in audit conclusions based 
on evidence of sufficient detail, it is evident that enhancement of these benefits is 
achieved only at the cost of performing more extensive audit procedures. Auditors 
should explicitly consider, at least in rough qualitative terms, this cost/benefit 
relationship when determining what constitutes an adequate level of evidence for a 
particular assignment. In such considerations, the internal auditor should strive to 
achieve a basic understanding of the user's expectations concerning the audit. A 
careful determination of what users would like to have or, at least, would be prepared 
to accept in terms of audit results will help the auditor in resource allocation 


decisions. 
Summary 
In Section Three the discussion has centred upon what is meant by sufficient, valid 


and relevant evidence and what factors influence an auditor's assessment as to 


whether or not an adequate level of evidence has been gathered. 
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SECTION FOUR: AUDIT EVIDENCE EVALUATION 


To ensure that the adequacy of evidence in fulfilling audit objectives is properly 


evaluated, the auditor should consider the following: 


t Be aware of the assumptions which help structure the chosen audit 
approach. Various assumptions exist in audits, allowing auditors to 
perform their tests in an efficient manner. Some of these assumptions 
include that: certain items under review are material or immaterial; 
certain types of evidence or audit procedures are more reliable than 
others; the degree of past errors in the control system under review 
influences the probability of current errors; and special audit procedures 
to test for collusive fraud, forgery or elaborately concealed defalcations 
are not expected. An auditor should maintain a sense of reasonable 
skepticism towards the validity of these assumptions. Faulty assumptions 
may cause the auditors to omit from their review, items of significance. 
The discovery of faulty assumptions will normally dictate the need for 


additional evidence. 


e Ensure that the act of audit observation does not temporarily change 
the nature of the control activities subject to review. The occurrence 
of such a situation is called "observer contamination" and involves, 
typically, increased conscientiousness in the performance of control 
duties by persons aware of the fact that their activities are being 
observed by the auditor. Such contamination seriously reduces the 


value of any evidence related to the activity under review. 


2 Explicitly consider the auditability of the area subject to review and 
the need for any specialized skills to properly complete the audit 
assignment. Where such skill is required, the auditor will not be able to 
assess properly the adequacy of evidence gathered in fulfilling audit 
objectives without the addition of subject-matter expertise to the audit 


team. 
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€ Ensure that observational errors have not arisen as the result of bias or 


from misapplication of audit procedures. 


® Exercise professional judgment carefully, guided, where practical, by 
objective decision aids. Observational errors due to poor auditor 
judgment of required audit procedures, and improper audit conclusions 
resultant from invalid projection of test results, always constitute risks 
to the proper use of evidence which has been accumulated to fulfill 


audit objectives. 
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CONCLUSION 


Few auditors would dispute that an understanding of the subject of audit evidence 

is central to an effective audit process. While agreement exists as to the importance 
of the subject, an explanation of what constitutes sufficient and appropriate evidence 
and how it might best be collected is generally not available in the literature written 


for public sector auditors. 


The lack of information on audit evidence has resulted in greater reliance upon 
auditor judgment when determining what constitutes the proper gathering and 
evaluation of evidence. While this document clearly does not refute the importance 
of auditor judgment in all assignments, it has attempted to reduce the degree of 
auditor subjectivity by making explicit the factors and techniques which should be 


considered for all evidence decisions. 
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Appendix A 


A COMPARISON OF EXTERNAL AND INTERNAL AUDIT THEORY 


This Appendix illustrates through comparison of the external and internal audit 
models that only a marginal difference exists in their underlying conceptual frame- 
works. Because of their similar nature, we have concluded that it is generally valid 
to borrow and adapt external audit concepts to the internal audit context in areas 
where it is practical and useful to do so. The extrapolation of external audit 
concepts is particularly beneficial to internal auditors where it is evident that the 
external audit model has been subject to more extensive research and development 


than its internal audit counterpart. 


For example, we have found it useful to use external audit theory as a basis for 
establishing in the internal audit context, the position of audit evidence relative to 
other audit concepts. In addition, external audit approaches for determining how 
to properly gather and evaluate evidence can be usefully reformulated to fit with 


the internal audit process. 


This Appendix begins with a summary of external audit theory. The similarities 
and differences between this model and the internal audit model! are then presented 
to support the claim that it is valid to adapt external audit concepts to the internal 


audit context. 


A Summary of External Auditing 


External auditing is primarily concerned with the degree of correspondence between 
various assertions made in financial statements and established criteria. Financial 
statement assertions include, for example, that the enterprise subject to audit has, 
in fact, cash, receivables, inventories, etc., and that these items have been properly 
described and disclosed in the enterprise's accounts. Established criteria represent 
the standards through which the auditor evaluates whether the assertions are valid, 
and in external auditing these criteria are represented by generally accepted 


accounting principles. 
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External auditors are not required to ensure that financial results are reported in a 
mechanically exact manner; rather the external auditor seeks a fair presentation of 
results. This concept gives rise to the idea of materiality, or degree of precision 
required in financial reporting which will satisfactorily meet the needs of various 
users of such reports. Where "material" departures from the norms of the evaluative 
criteria exist, the external auditor is required to report on both the nature of the 


deficiency and its effect on the financial statements. 


Included in external audit theory is also the concept of verification. This concept 
refers to the gathering and evaluation of evidence which allows the auditor to 
express an opinion concerning the fairness of financial statement assertions. The 
external auditor does not strive for absolute certainty respecting the validity of 

the contents of the audit report; rather a reasonable basis is sought. In this sense, 
not all evidence available is accumulated to support the contents of the audit report. 
Instead, the auditor determines what constitutes "sufficient and appropriate" 


evidence in addressing the needs of a particular audit. 


Similarities and Differences 


Despite significant differences in terms of the uses of the audit report, the subject 
matter reviewed and the evaluative criteria employed, there exists considerable 
similarities in the basic nature of internal and external auditing. In identifying the 


major similarities the following points should be recognized: 


® Although the breadth of the subject matter under investigation in an 

internal audit usually differs from that of an external audit, the role of 
the auditor in both cases is essentially the same. Similar to the external 
auditor's role, the internal auditor attempts to establish the degree of 
correspondence between the subject matter under review and certain 
predetermined evaluative criteria. Whereas the subject matter of external 
auditing is specifically focused on financial control and reporting, the 
internal auditor's subject matter includes study of the adequacy of all 


controls in the area under review, including financial controls. 
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& Like external auditors, internal auditors do not seek exact correspondence 
between the actual auditee conditions under review and the standards 
of performance represented within the evaluative criteria. Below some 
level of detail relating to the adherence of auditee operations to 
established performance standards, auditors show little concern. Similar 
to the external auditor's concern for fairness as opposed to mechanical 
exactitude, the internal auditor seeks adequacy in operations, not 
perfection. The threshold of what constitutes a significant deviation 
from acceptable system behaviour gives rise to the need for a concept 


of materiality for internal auditors. 


e Both internal and external auditors are required to report on significant 
departures from the standards embodied within the evaluative criteria. 
External auditors are required to disclose significant departures from 
generally accepted accounting principles, and internal auditors are 
required to disclose significant departures between actual system 


behaviour and the desired behaviour embodied in their criteria. 


e Both internal and external auditors are obligated to seek out additional 
evidence for those areas where deficiencies are found. The external 
auditor uses the additional evidence to determine the effects of the 
deficiency on the financial statements. The internal auditor uses 
additional information on the causes and effects relating to the noted 
deficiency so as to illustrate the nature of the problem, its general 


significance and appropriate action to correct the problem. 


e Internal auditors are bound to conduct examinations and verifications to 
a reasonable extent and the internal audit cannot give absolute assurance 
that non-compliance or irregularities do not exist. This requirement for 
reasonable assurance mirrors external audit practice. Internal auditors 
are faced with the need to select from available evidence that which 
constitutes sufficient, valid and relevant evidence for their particular 
purposes on any given assignment. In Section Three, the decision rules 
that external auditors use in determining the extent of evidence that 


must be gathered have been adapted for use by internal auditors. 
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Review of these similarities between internal and external audits reveals that many 
of the underlying audit evidence concepts are essentially the same, and that internal 
auditors may benefit by structuring their understanding of the audit process using 


the conceptual framework developed for external auditing. 


While it is useful to draw from external audit theory wherever it is practical and 
appropriate do to so, internal auditors should also be aware of certain fundamental 
differences in the nature of the two types of audit. While these differences do not 
refute the conceptual framework put forward here, they do have an impact on the 
manner in which the internal versus external audit process is performed. The major 


differences include: 


° the degree to which the auditor will approach users of the audit services 


and the nature of the relationship between them; 


e the breadth of the subject matter reviewed during the audit (audit scope) 


and its effects on the audit process; and 


e the nature of the evaluative criteria used in the audit. 


Each of these areas is discussed below. 


Users of Audit Services 


In this document, the term "user" refers to the primary intended recipient of the 
audit report and the person for whom the auditor performs the audit services. The 
user is to be distinguished from the "auditee" who represents the manager responsible 


for the operations and activities which are the subject of the auditor's examination. 


In general, the user of the internal auditor's services is much more readily 
identifiable than users of an external audit. In the federal government, the internal 
auditor typically performs duties for a deputy minister, or equivalent for agencies, 
or senior officials. Secondary users of internal audit reports would include 
departmental or agency managers subordinate to those noted above and external 


audit or central agency review groups who may wish to rely on the work performed 
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by the internal auditor. The external auditor, on the other hand, acts on behalf of 
the audited entity's "shareholders" and more generally to all readers of the 


enterprise's financial statements. 


The fact that the internal auditor reports to a particular "user" is significant in 

that it provides the auditor with an opportunity of receiving input concerning the 
nature and extent of testing that would satisfy the user's needs. In external auditing, 
such input is essentially non-existent; it is only through the external auditor's 
professional associations that guidelines and standards are developed to aid in 
determining answers to problems relating to the nature and extent of audit testing 


required. 


Subject Matter Differences 


The subject matter of external versus internal audits represents an obvious area of 
difference. Whereas external auditors are primarily concerned with the fairness of 
financial statement presentation, internal auditors are also concerned with the 
economy, efficiency and effectiveness of all internal management policies, practices 


and controls. 


The knowledge requirements of internal auditors will vary from external auditors 

as a result of differences in the subject matter under review. An internal auditor, 
for example, must have a broad understanding of management and operational 
controls and reporting procedures; an external auditor, on the other hand, requires 

a specialized knowledge of financial control and reporting. While these differences 
in subject matter skill should exist between internal and external auditors, it is the 
contention here that the methods by which both analyze their subject are essentially 


the same. 
Nature of Evaluative Criteria 


Both internal and external auditors seek to establish the degree of correspondence 
between the subject matter under review and predetermined evaluative criteria. 
Obviously the criteria used by external and internal auditors will differ due to the 


varying subject matter under review. Less evident, however, are differences in 
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terms of the degree of latitude allowed in the determination of what constitutes 

the appropriate criteria for an audit. In external auditing, the established criteria 
are widely accepted and codified by the Canadian Institute of Chartered Accountants 
in the form of generally accepted accounting principles. In internal auditing, 
evaluative criteria used to judge the adequacy of auditee operations have not been 
codified. To a large extent, these criteria are currently chosen on a judgmental 
basis by the internal auditor, typically supported by reference to authoritative 


sources and information derived from discussion with users and auditees. 


Given this perspective on the nature of external and internal audit criteria, the 
major difference to note is the degree of latitude allowed in their determination. 
In external auditing, criteria are relatively fixed while in the internal auditing 
considerable flexibility may be allowed. Indeed, it may be that internal audit 
criteria will never reach the status of "generally accepted" but perhaps will require 


ongoing determination on a situational basis. 


The fact that internal audit criteria are not as fixed or widely accepted as external 
audit criteria may give rise to differences in the manner in which the audit is 
conducted. Whereas it is rarely that external auditors would question the validity 
of the criteria which establishes the basis of their evaluation, it is quite plausible 
that internal auditors may find their criteria deficient in light of additional 


information which has come to their attention during the course of the audit review. 


Summary 


We have attempted to establish the major similarities and differences between 
external and internal auditing. While on the surface external audits appear quite 
different from internal audits because of their differing subject matter, criteria 
and uses, it is evident on closer examination that the two types of audit differ only 
marginally in terms of the analytical methods which provide the real substance to 
audit evidence theory. Because of their similar nature, we have concluded that 
there exists much potential for the borrowing and adapting of concepts between 


the two types of audit. 
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CHAPTER 6 


AUDIT JUDGMENT, DECISION SUPPORT AND EXPERT SYSTEMS 


INTRODUCTION 


This chapter consists of three sections. Section One summarizes what is currently 
known about human problem-solving behaviour, focusing especially upon the 
limitations of human judgment and decision processes, including those of professional 
problem solvers such as auditors. The main point of the material is that structural 
aspects of human cognitive (thinking) processes impose rather severe limits on 
human problem-solving behaviour. These limits lead to reliance upon judgmental 
heuristics (i.e., short-cuts and rules-of-thumb). The heavy reliance upon judgmental 
heuristics leads to systematic judgmental biases which adversely affect the quality 


of professional judgments and as a result, limit the quality of solutions. 


Section One also describes the nature of professional expertise, focusing on the 
nature, use and acquisition of knowledge, a necessity for coping with complex and 


difficult problems. 


Given the limits of unaided human problem-solving as outlined in Section One, 
Section Two discusses principles of judgment enhancement that are relevant to 
auditors. This section includes coverage of planning aids, information gathering 

aids, documentation aids and information evaluation aids. Each sub-section describes 
several techniques for enhancing the ability of problem-solvers to deal with complex 


problems, or for reducing the deleterious effects of heuristics and biases. 


The discussion in Section Two leads to a consideration of automated decision support 


and expert systems described in Section Three. 


The contents of this chapter complement the contents of the preceding Chapters 4 
and 5, dealing with analysis concepts and practices and with audit evidence 


respectively. 
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SECTION ONE: LIMITS OF HUMAN PROBLEM-SOLVING 


Cognitive psychology treats problem-solving as the manipulation of an internal 


representation of an external environment (Hunt 1983). 


When a person is given a problem to solve, unless it is of a completely 
familiar kind, he must first determine what the Problem is, understand 
it fully, and find some way of representing it to himself before he can 
go to work on it and seek a solution by heuristic search (Simon 1977, p. 
74), 


In keeping with this widely accepted model of problem-solving, we would expect an 
auditor attempting, for example, to prepare an audit plan to construct internal 


representations of the: 


8 initial problem situation -- in this case, his or her view of the assigned 
task of preparing the audit plan; 


8 goal situation -- his or her view of what an ideal or satisfactory solution 
would be; and, 


& operations available for moving from the first to the second 
representation. 


In some instances, particularly in complex situations, such operations may not be 
readily apparent, so the goals or the problem itself may be redefined by the auditor 
to permit a solution to be achieved, although this may not be a solution to the 
problem that was originally posed (Wright 1974, Shields 1980, Choo and Eggleton 
1982). 
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As is apparent from the foregoing, the means of representing aspects of the problem 
situation figure prominently in the characterization of problem-solving, which has 
been described as finding the representation of a problem which would make the 
solution obvious.! A person's choice of problem representation determines the 
strategies available for problem-solving (Hunt 1983), which is commonly sub-divided 


into the following phases: 
@ setting goals, 


@ acquiring information about the task environment, in particular about 


problem structure, 


® integrating systems of information (i.e., evaluating information and 


formulating judgments), and ultimately, 


® choosing solutions (and acting upon them). 


Certain aspects of problem-solving have been researched extensively for several 
decades, and some generally accepted findings are available, although for various 
reasons many of these findings have not found wide application in professional 
problem-solving environments (Slovic et al., 1977). These findings have two main 


aspects. 


® First, there are common traits shared by all problem-solvers, professionals 
and laypersons alike, pertaining to structural features of human 


information processing such as: 


1 Boritz (1981) discusses the potential importance of problem representation 
techniques in contributing to improved problem-solving by auditors evaluating 
complex systems. He hypothesizes that certain information-acquisition and/or 
structuring techniques, governed by the particular problem representation 
scheme used, may enhance a reviewer's understanding of the original auditor's 
problem-solving behaviour and conclusions, permitting better feedback from, 
and control over, the evaluation process; hence, better audit outcomes due to 
improved review and supervision, even if the techniques used did not directly 
improve the original evaluator's judgments. Some of. these techniques are 
discussed in Section Two of this chapter. 
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- the strycture, capacity and capability of short-term memory 
(STM), 


- the nature of elementary information processing functions, and 


- the central role of rules-of-thumb (heuristics) and judgmental 
biases in support of information acquisition, evaluation and 
judgment formulation. 


® Over and above these traits, there are features which figure prominently 
in distinguishing between experts and novices in professional fields 


including: 


- the nature and structure of knowledge in a particular domain 
(e.g., EDP audit domain) in long-term memory (LTM), and 


- the methods used to gain an understanding of a problem. 


In summary, the problem-solving behaviour of experts may be contrasted with that 
of novices along the dimensions of knowledge of facts and problem-solving strategies 
and methods; however, generalized cognitive aspects of problem-solving behaviour 
also play a prominent role in determining the quality of solutions. Cognitive aspects 
of problem-solving act as constraints upon all problem-solvers and lead to the 
emphasis upon enhancements of, among other things, problem representations as a 
basis for achieving or improving the quality of solutions for all levels of expertise. 

In the next four sub-sections, cognitive traits are discussed in more detail and some 
of their implications are highlighted. First, structural aspects of cognitive processes 
are discussed. Next, heuristics and judgmental biases are covered. Third, a discussion 
of expert knowledge is provided. Finally, a model of professional problem-solving 
which integrates the discussion is presented. The purpose of this discussion is to 
introduce the general concepts and findings which have a direct bearing on the 


development of decision support and expert systems. 


N 


Both short-term and long-term memories are conceptualizations, not physically 
identifiable parts of the brain. Short-term memory is the part of the mind 
where problem-solving activity is considered to take place (i.e., a scratch 

pad). Long-term memory is the part of the mind where knowledge is considered 
to be stored, organized and developed (i.e., the encyclopedia). 
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Structural Aspects of Human Cognitive Processes 


The human information processing model of cognition draws upon the computer for 
its central metaphor - characterizing cognition as a series of input, processing and 
output operations aided by auxiliary storage facilities and elementary information 
processing functions. Cues are pieces of information which are collected from the 
problem area and from its environment by sensory mechanisms and delivered into a 
part of the auxiliary storage area called short-term memory, where a mental image 
of the problem is created. Similarly, knowledge stored in another part of auxiliary 
storage called long-term memory is retrieved by the elementary information 


processes and deposited into the short-term memory (working storage) area. 


In this view of human behaviour, emotions, motivations and personality traits are 
de-emphasized. Primary attention is paid to the way in which people perceive 
data, integrate information and make judgments, in short, how people process 


information. 


In a review of the relevant research, Simon (1979, p. 386) summarized the important 


structural characteristics of human cognitive processes as follows (refer to Figure 1). 


Cue Recognition 


The cue recognition process, called the evocative mechanism, is quite rapid, 
requiring 3-500 msec.” to access information in long-term memory; but, it depends 
upon the presence of familiar stimuli, or cues. In other words, problem-solvers 
recognize rather than discover, features in a problem (e.g., in a review of an 


internal control system). 


—— 


3 1 msec. = 1/1,000,000 of a second. 
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THE HUMAN INFORMATION PROCESSING MODEL OF COGNITION 


Short-Term Memory 


Capacity: “4 chunks” 
Retrieval: 3-500 msec. 


Update: 3-500 msec. 


Elementary 
Information 
Processing 


Functions 
Long-Term Memory 


Capacity: Unlimited 


Retrieval: 2 sec. 


Update: 8 sec. 


Figure 1 
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Knowledge (Long-term Memory) 


Knowledge is stored in long-term memory, which may be viewed as an associative’ 
memory with virtually unlimited capacity. Access to stored information for purposes 
of retrieval requires about two seconds, while updating is quite a bit slower, 
requiring about eight seconds per familiar chunk.? No significant learning takes 
place in tasks requiring less than one hour. In other words, many types of experience 


may not be experience at all if they have not been stored in the knowledge base. 
Working Storage (Short-term Memory) 


Short-term memory represents "working storage" for information chunks, the place 
where problem-solving actually takes place. It is characterized by fairly rapid 
retrieval and storage times of 3-500 msec. per chunk; but, very limited capacity, 
about four chunks in all. These characteristics permit only limited amounts of 
information to be used in a problem-solving search or hypothesis-generation at any 
given time. Since the amount of information that can be used is limited, its quality 


becomes an increasingly important consideration. 


Many variables can influence behaviour and the quality of problem-solving that 

may result, including various personality traits, aspiration levels, motivation and 
incentive schemes, and so on. However, of all the potential limitations upon problem- 
solving, the most severe, and for all practical purposes the least flexible, is that 
imposed by the structural nature of short-term memory. Thus to improve task 
performance in complex problem settings may require the development of techniques 
specifically aimed at compensating for the constraints imposed by the structural 
characteristics of short-term memory. There are essentially two strategies for 


accomplishing this development. 


4 In an associative memory, data are made accessible by the content of the data 
being retrieved rather than by its physical location. 


5 A chunk is any organization of information that has previously become familiar 
(Simon 1979, p. 368). Thus, a chunk may be an individual element of information, 
or a pattern of many elements combined into a single unit for purposes of 
information processing. 
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* First, external short-term memories (e.g., computerized "scratchpads") 
with greater capability may be developed. This is the approach inherent 
in many automated support systems proposed as aids to professional 
problem-solvers. 


& A second strategy focuses on the nature of chunks with the aim of 


assisting problem-solving by making each chunk contain more, and better, 


information or knowledge. This is the approach inherent in the use of 
training, procedural guidelines and expert systems. 


Heuristics and Biases 


Research findings enumerate a variety of heuristics and biases which determine to 
a large degree how information is acquired and how it is evaluated.° Although 
often considered together, heuristics and biases may be usefully distinguished. 
Both represent potentially severe limitations on the quality of final judgments or 
solutions to problems; however, in general, a heuristic should lead to judgments 
being randomly distributed around some ideal "correct" value, while a bias would 
lead to judgments systematically falling on one side of the ideal.’ Biases may be 


viewed as the products of heuristics or short-cuts which fail in systematic ways. 
Heuristics 


A heuristic is a procedure which may solve a problem, but offers no guarantee of 
accuracy. In contrast, an algorithm, if correct, and if given sufficient time and 
resources, will always produce a correct solution. For exarnple, suppose one wished 
to determine an employee's annual salary given an hourly pay rate of $10.00. An 
appropriate algorithm would accumulate the hours worked for Jan. 1, Jan. 2 ..., 
Dec. 31 and multiply by $10.00 (assuming no overtime) to arrive at the employee's 
annual salary. In contrast, a reasonable heuristic might be to multiply 50 weeks by 


40 hours per week by $10.00 per hour, arriving at an estimate of $20,000. This 


6 Hogarth and Makridakis (1981) reviewed and summarized a large number of 
behavioural studies emphasizing the flaws inherent in planning and forecasting 
activities. Mock and Vertinsky (1985) discuss some documented problem 
solving flaws as related to risk analysis in auditing/accounting. 


7 Of course, several different biases could lead to judgments falling on opposite 
sides of the ideal value, and thus they could cancel themselves out. 
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estimate would rarely achieve the accuracy of the algorithm; however, for most 
purposes, it would yield a reasonably close estimate. Of course, many situations 

are far more complex and the use of heuristics, sometimes called rules-of-thumb or 
short-cuts, may not yield estimates of an adequate quality (i.e., accuracy). However, 
they are valued, sometimes inappropriately, because they reduce the amount of 

time or effort required to come up with satisfactory solutions. Heuristics, then, 

are problem-solving short-cuts affecting the way in which problems are viewed and 


solved. Researchers have identified three main types of short-cuts: 


® Means-ends analysis (Simon 1969) is used to navigate through a problem 
space, affecting the thoroughness of information search. 


® Elimination-by-aspects (Tversky 1972) is used to reduce the 
dimensionality of a given problem, affecting the number of factors 
considered in evaluating alternatives. 


8 Computational short-cuts are used to simplify otherwise complex 
algorithms, affecting the accuracy of solutions obtained. 


Means-ends analysis: Means-ends analysis is one of the first heuristics identified 
by researchers and is described as follows: 


Given a desired state of affairs and an existing state of affairs, the task 
of the problem-solver is to find the difference between these two states 
and then to find the correlating process that will erase the difference. 
(Simon 1969, p. 112) 


Thus, for example, given a model of a well-controlled system, and a representation 
of the client's actual system, an auditor would follow a selective process of 
identifying the differences and the control improvements which would eliminate 


them. 
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The implication of this heuristic for auditors is that the extent of information 
search will, in the absence of compensating mechanisms, depend heavily on prior 
experience, and will not necessarily obtain, in any reasonable time frame, the 
complete or appropriate information to bear upon a particular problem. The 
heuristic implies satisficing,® rather than optimizing behaviour in the search 
process. Furthermore, even if an inappropriate goal state is selected by the auditor, 
that goal state will nevertheless guide the problem-solving behaviour. Also, if the 
current state is not accurately represented, the starting point for the search for a 
solution will be inappropriate. Finally, if the laws for linking the starting state and 
goal state are not well understood, the path traveled in the information search 


process (i.e., the line of reasoning) will likely be unacceptable. 


Elimination-by-aspects: Tversky (1972) described choice as a covert elimination 
process. When faced with a multi-dimensional decision problem, the problem- 
solver is assumed to select a dimension or aspect.” Then, all the alternatives which 
do not possess that aspect are eliminated. If more than one alternative remains, 


the procedure is repeated until all but one of the alternatives are eliminated. 


The implication of this heuristic for auditors is that, in the absence of compensating 
features, the number of alternatives considered and the number of dimensions used 


for making comparisons and evaluations will be very small. 


Computational short-cuts: An early set of studies (Einhorn 1972, 1974) suggested 
that while expert professionals were quite proficient at recognizing and diagnosing 
individual items of information (cues), they were poor at combining the individual 
cues into accurate global judgments. For example, suppose the goal is to discover 


T. By studying the repeated judgments of experts, a model is developed (e.g., by 


8 Satisficing means taking the first satisfactory solution that "comes along". In 
contrast with optimizing, which implies searching the entire set of feasible 
solutions to find the best, a "pure" satisficing search will never search past 
the first feasible solution. 


9 This selection could be based on: random selection; systematic evaluation of 
importance relative to all other alternatives; biases such as availability; or, 
external requirements, such as procedural guidelines, standards or rules. 
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regression analysis) suggesting that T is an additive function of A, B and C. 
Observations A°, B° and C°, are made and then T is predicted by T', a guesstimate 
ot the true status of T. Over two decades of research would predict that while a 
professional problem-solver would probably do quite well at observing A°, B° and 
C°, his/her guesstimate of T, T' would be quite unreliable. Indeed, a simple rule 
such as: "Add XA° + YB° + ZC° to get T', where A°, B® and C° are observations 
made by a professional and X, Y and Z are estimated coefficients from a model 
based on that professional's own past performance, would consistently do better at 
predicting T than if the professional were asked for T directly. 1° On the basis of 
findings such as these, Dawes (1974, 1979) concluded that the role of experts should 
be to determine which variables should be used, the direction of their effect, and 
to make observations of them, but leave the combining process to mechanistic 
rules or models, because such models will virtually always outperform the experts 


themselves. II 


Biases 


Once information is acquired, evaluative judgments are made to permit inferences 
and conclusions to be drawn. In recent years, research findings have pointed out a 
number of recurring flaws in judgmental/inferential processes which interfere with, 


and systematically bias, the conclusions reached. 


In a now classic and widely cited series of research studies, Tversky and Kahneman 
(1974) provided the foundation for much of the recent research into judgment and 
inference. They identified three sets of inferential biases resulting from heuristics 
that they termed representativeness (judgment by stereotypes rather than objective 
data), availability (judgment by ease of access to data rather than thorough 
information search) and anchoring and adjustment (judgment by precedent rather 
than independent assessment). Fischoff (1975) identified the hindsight bias which 
suggests that in looking back at past judgments, problem-solvers find ways of 
rationalizing their behaviour, using what they know now rather than what they 


knew then. Consequently, they learn less from feedback than they should. 


10 Even if X, Y and Z are simply replaced by I's. 


11 These models can be more complex than simple addition rules and they can 
combine different specialists, judgments on a variety of different variables. 
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These flaws in judgment processes are considered to be inherent structural 
characteristics of the way in which humans encode experiences and the manner in 
which individuals make judgments. If they are structural in nature, these limitations 
cannot be corrected by mere changes in the incentive structure or by repeated 
experience. Nor are they characteristic of laypersons alone; but rather, are equally 


typical of professionals, experts, and so on. 


Representativeness: The representativeness heuristic predicts that judgments 
about observations will depend heavily upon their similarity to other, previously 
encoded observations, and not on their objective representativeness of facts. Some 


of the biases included under this umbrella term are: 


- insensitivity to prior probabilities of outcomes (i.e., the objective base- 
rate frequencies of outcomes); e.g., the objective base-rate probability 
for drawing an ace of spades from a fair deck is 1/52, about 2 per cent; 


- insensitivity to sample size (i.e., attributing equal value to small and 
large samples); 


- misconceptions of chance (i.e., holding false assumptions about the 
behaviour of random events); 


- insensitivity to predictability (i.e., basing predictions on unreliable or 
unstable evidence); 


~ the illusion of validity (i.e., overemphasizing correlated variables despite 
the predictive superiority of uncorrelated variables); 


- misconceptions of regression (i.e., failing to appreciate that extreme 
observations will be followed by less extreme ones, naturally, with a 
high probability). 


Availability: The availability heuristic predicts that people will make judgments 
about frequency of a class of events on the basis of the ease with which similar 
instances or occurrences can be brought to mind, and not in accordance with their 
objective importance. Some of the biases included under this umbrella term include 


biases due to: 
- retrievability of instances (i.e., overgeneralizing about a class on the 
basis of one salient instance in memory); 


~ effectiveness of a search set (i.e., overgeneralizing from knowledge 
retrieved on the basis of easily used search arguments); 
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- imaginability (i.e., overgeneralizing from easily imagined hypothetical 
events); 


= illusory correlation (i.e., attributing common causes to items associated 
in memory structure). 


Anchoring and adjustment: The anchoring and adjustment heuristic predicts that 
people will make estimates by first focusing on an initial value and adjusting from 


that value to reach a solution. Some of the biases related to this heuristic include: 


- insufficient adjustment from an initial estimate, 


- overestimation of probabilities of conjunctive events (i.e., unwarranted 
optimism about success), 


- underestimation of probabilities of disjunctive events (i.e., 
underestimation of probabilities of failure in complex systems), 


- anchoring in the assessment of subjective probability distributions (i.e., 
expressing more confidence about items than is warranted by actual 
knowledge). 


Hindsight: The hindsight bias predicts that people, looking back at events that 
have happened, will exaggerate what could have been anticipated about them in 
foresight. They may even "misremember" their own predictions so as to exaggerate 
the predictability of events that have transpired (Fischoff 1982). This bias can 
result in overconfidence in one's predictive ability as well as insensitivity to causes 
of errors or deviations from planned courses of action, since there is a tendency to 
conclude that more could have been anticipated, hence controlled, than was in fact 


possible at critical decision points in the past. 
Fischoff (1982) summarizes the key features of biases as follows: 


& Biases result from, "the confrontation between a deterministic mind 
and a probabilistic environment"; i.e., they are typical of processes 
which involve uncertainty and require probabilistic inferences to be 
made about them by the problem-solver. 


é Problem-solving flaws are most severe in the information integration 
phase of problem-solving rather than the information identification 
phase. 
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® Biases are due to cognitive limitations and are not due to misinformation 
or deception. 


« Biases are measured relative to generally accepted norms for evaluating 
judgments. They are not concerned with varying preferences among 
individuals or for a given individual over time. 


® Biases are the result of human intuitive judgment rather than artifacts 
of information systems. 


4 Biases are cognitive, not emotional in nature. 


It is important to recognize that many erroneous judgments, inferences and decisions 
will not in themselves automatically result in severe negative consequences. Einhorn 
and Hogarth (1981) and Hogarth (1981) emphasize that a number of environmental 

or situational factors mitigate against such severe consequences. The most prominent 
of these, cue redundancy and feedback obtained through social interaction, permit 
tentative judgments to be made and then revised, perhaps several times, prior to a 
final conclusion or choice being made. However, Fischoff's (1982) research suggests 
that professionals often tend to treat certain findings as if they "knew-it-all-along". 
This "hindsight" bias may prevent learning from mistakes and inhibit corrective 
action, even when feedback is available. Related research has consistently found 
that people are more confident of their judgments than their actual skill should 
permit them to be, even given feedback. Thus, although there has been much 
speculation about the role of feedback, its actual value in practice is neither clear 


nor well watapiened.+~ 


12. Reviewers can be hypothesized to play an important, even critical, role in 
providing corrective feedback about the initial evaluations made by auditors. 
Since, typically, reviewers are more expert than the original auditors who 
perform the initial work (e.g., initially evaluate the system of internal 
control) an interesting researchable question emerges about the way reviewers 
revise audit plans presented for their evaluation. Some research suggests 

that reviewers, in addition to 'suffering' from the same cognitive limitations 
as reviewees, tend to anchor on the work they are reviewing, making 
insufficient corrective adjustments. 
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The Nature of Expertise 


Expertness in a professional domain is related to both the amount and the kind of 
knowledge stored in long-term memory and the efficiency of the evocative mechanism 
when responding to perceptual cues. Novices tend to process facts as individual 
elements. Thus, when the number of elements being processed exceeds an individual's 
short-term memory (STM) capacity, they begin to "drop off the edge", and information 
is lost. In-contrast, experts in a given domain recognize a small number of patterns 

in a larger whole which they retain as chunks; thus, they are able to derive more 


meaning from a given set of environmental cues than can novices. 


The combination of individual elements of information into composites or chunks 
representing patterns of elements is a defining characteristic of expertise in a 
given domain and is considered to depend upon two variables which can, at least 
partially, be controlled by professionals: professional training and field experience. 
The development of expertise is a slow process; a way of compensating for problems 
that may arise from the slowness of development is to embed the knowledge of an 


expert within a set of procedures especially designed to be followed by novices. 


Experts are postulated to have databases of thousands of patterns with a structural 
organization conducive to ease of access. Thus, their knowledge can be brought to 
bear upon specific problems. In contrast, novices not only have a smaller reservoir 
of knowledge to draw upon, in terms of the absolute number of patterns, but also, 
and equally important, they have weaker structural interweaving of those patterns, 
affecting their ability to link knowledge with evocative perceptual cues (Simon 
1979). This in turn affects both their ability to retrieve knowledge during problem- 


solving and also their ability to update old knowledge with new facts. 
In distinguishing the expert and novice medical diagnostician, Johnson et al., state: 


Contrary to the novice, disease knowledge of the expert is both precise 
and richly detailed. Through clinical experience the internal structure 
of experts...is "tuned" to the natural variation in findings. Such tuning 
generally allows the expert to properly interpret findings for a case 
that novices do not. Because of additional training as well as extensive 
experience, the expert also has a hierarchy of disease knowledge that is 
0 eae and extensively differentiated. (Johnson et al. 1981 

p. 
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In much of the research dealing with expert knowledge, the ultimate goal is to aid 
professional problem-solving by transferring the skills of the expert professional to 
the novice by codifying, where possible, various aspects of expert knowledge into 


the procedures to be used by the novice. 


In brief, knowledge is a key factor determining the quality of professional problem- 
solving. In addition to the absolute quantitative difference in the amount of 
knowledge possessed, the qualitative difference in the problem-relevant knowledge 
possessed by experts and novices distinguishes the quality of their respective 
solutions to problems. By focusing on the important differences between novices 

and experts in a particular problem-solving situation, a researcher might identify 

the gaps in knowledge which, if filled, might help novice professionals avoid 
committing serious errors. Eventually, critical knowledge might be built into expert 
systems; i.e., automated systems supporting the problem-solving behaviour of novices 


by mimicking experts. 


There are three main aspects of expert knowledge which are of interest: 


g the nature and structure of professional expertise (expertise = knowledge); 
a the use of professional knowledge; and 
9 the acquisition of professional knowledge. 


Expertise=Knowledge 


The prevailing view is that there are essentially two categories of knowledge: case 
knowledge (Elstein et al. 1978) and systems knowledge (Feltovich 1978, Simon 

13 
1979). 


13. Feltovich (1978) describes the nature of medical case knowledge, but it is 
assumed here that the description is equally applicable to the accounting/audit 
context. It is an open issue whether experts in accounting/auditing are the 
same kind of experts (i.e., cognitively) as chess players or doctors. 
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Case knowledge: Case knowledge forms a large portion of any professional's 
storehouse of information. Thus, case knowledge: (i) includes knowledge about 
which environmental! information cues are signals about which problems, errors, 
etc., (ii) is organized from the bottom-up, whereas basic accounting/auditing theory 


is organized from the top-down, and (iii) is used to generate diagnostic hypotheses. 


Some case-based information items or cues correspond exactly to unique facts, 
hence such knowledge can help discriminate among several potential explanatory 
factors and permit conclusive isolation of the correct hypothesis. Other cues 
correspond to classes of facts. They can serve to discriminate among classes of 


hypotheses; but, cannot by themselves isolate a unique hypothesis. 


Of course, case knowledge is often less than ideal. In particular, since case 
knowledge is integrated with general domain knowledge, in cases where domain 
knowledge is restricted (e.g., an auditor has no knowledge of the domain of on-line 
computer systems), the case knowledge will be overly specific and insufficiently 
generalized. In addition, since novices! causal links are often insufficient (e.g., an 
auditor may have no knowledge about many of the various causes which might explain 
a given control weakness), novices would tend to focus on a restricted sub-set of 


possible, but, in the given circumstances, incorrect hypotheses. 


Systems knowledge: Systems knowledge refers to knowledge about the principles 

of normal behaviour of an "object", say an accounting system. Such knowledge may 
be used to reason abstractly about problems, detached from the particular 
information being observed. However, use of such knowledge is cognitively difficult 
or strenuous in comparison to the prototype-matching (i.e., matching against previous 
cases) exercise characteristic of the application of case knowledge which merely 
involves recognition of previously encoded "knowns" in a given situation. Thus, the 
use of systems knowledge is usually restricted to novel problems and generally is 
performed only by expert professionals, and then only in complex or novel problem 


situations. 
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The Use of Knowledge in Problem-solving 


Experts display a learned ability to search the problem environment for specific 

cues in the task environment. In addition, by focusing on cues in a hierarchical 

fashion (i.e., from general to specific), experts are able to reduce drastically the 

size of the problem (in a manner similar to a binary search) until a solution (e.g., 
diagnosis) is reached. Finally, search efficiency and diagnostic accuracy are positively 
correlated with the amount of experience. More experienced physicians ask fewer 
questions and make fewer diagnostic errors, are better able to selectively discard 


irrelevant data, and to better retain only decision-relevant data (Kleinmuntz 1968). 


Elstein et al. (1978) found that accuracy of cue interpretation was related to 

accuracy of diagnostic outcomes, but independent of thoroughness of cue acquisition. 
Thus, they concluded that thoroughness of data collection and accuracy of cue 
interpretation represent two separate dimensions to be studied. In addition, they 
suggested that lack of thoroughness is not as important a cause of error in medical 
diagnosis as problems of integrating and combining information. This would appear 

to be consistent with Hogarth's (1981) hypothesis that the cue redundancy prevalent 

in most task environments tends to reduce the "opportunity" for errors of omission. 
When cues are ignored it is due less to oversight than to missing evocative connections 


in memory; i.e., either missing knowledge or missing access paths to that knowledge. 


More recently, Johnson et al. (1981) describe an intensive multi-method study of 
expert diagnostic reasoning in medicine. A series of three experiments was used to 
both explore and test hypotheses about expertise and error in medical diagnosis. !* 


Findings from the three experiments indicated the following: 


14 ‘In the first experiment, three groups of subjects were used: four experts, 
four trainees and four medical students. All were given the same patient 
data, and a process-tracing methodology was used to study the behaviour of 
subjects within the three levels of expertise. In the second experiment, a 
computer simulation model was used to identify conditions under which errors 
in reasoning discovered in the first experiment could be related to specific 
data cues provided in the case. Predictions derived from this analysis were 
then tested in a third experiment with a new sample of 12 subjects at three 
levels of expertise. 
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® The form of diagnostic reasoning was similar for all subjects trained in 
medicine, regardless of rank (and was successfully captured by a 
computer simulation model). 


e The substance of the diagnostic reasoning by experts was similar to that 
represented in the simulation model, but not that of novices; i.e., it was 
possible to distinguish the substantive knowledge of experts from that 
of novices. 


8 Errors in subjects' reasoning were attributable to deficiencies in disease 
knowledge and the interpretation of specific patient data cues (and 
were capable of prediction by using the simulation model). 


Johnson et al. identify three common errors in diagnostic reasoning by novices, 
resulting, they suggest, from the limited models used in medical training and 
relatively limited experience with any given set of laws, which lead to knowledge 


configurations with an internal structure that is fairly imprecise. These errors 


include: 

e being too tolerant; i.e., failing to reject an inappropriate hypothesis, 

a being too rigid; i.e., failing to entertain a correct hypothesis long enough 
because it is not precisely in line with expectations, even though it is 
within normal/allowable limits, 

® simply not thinking of the correct hypothesis. 


In Johnson et al.'s experiments, the less expert subjects exhibited a strong data- 
driven dependence in the flow of their reasoning behaviour; i.e., they seemed to be 
pushed from one hypothesis to another, depending on the most recent strong infor- 
mation cue that they observed in the data. In contrast, the more expert subjects 
followed lines of reasoning which incorporated the full set of competing hypotheses, 
although two different strategies were used, described as "breadth-first" or "depth- 


first" lines of reasoning. !? 


15 Ina breadth-first line of reasoning, termed a "precautionary" strategy, a 
number of alternative hypotheses are generated early then simultaneously 
entertained; so, if evidence which disconfirms a particular hypothesis is 
encountered, the subject can easily switch to an alternative. In contrast, a 
depth-first line of reasoning is characterized by the pursuit of one hypothesis 
until strong disconfirming evidence is discovered. Success in such a line of 
reasoning depends upon the diagnostician's ability to reject the current 
hypothesis, when appropriate, and to jump to a new one. This requires relatively 
precise knowledge. 
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These findings suggest that judgment enhancement should not look to improve 
professional problem-solving by manipulating the form of reasoning; instead, the 
emphasis should be upon the substance of reasoning, focusing on the factors which 
lead the expert to the right conclusion when the novice, following the same line of 


reasoning, errs. 


Of course, experts are not immune to errors. Even experts can err if data ina 
given case are missing or are not adequate to trigger the appropriate prototype, so 
that even though the correct model exists in memory, it is not considered. In 
addition, as mentioned previously, both experts and novices alike are limited in 
their capacity as information processors; hence, heuristics and biases are likely to 
affect the manner in which information is evaluated. Novices would tend to base 
their 'biased' conclusions on a smaller set of experiences, focus on less appropriate 
cues in the environment, and search ina smaller range of the problem space, while 
experts would display better, but nonetheless biased and erroneous, judgments; for 
example, when their prior expectations serve to overemphasize cues which conform 


to those expectations and underemphasize weaker cues which do not so conform. 


Acquisition of Knowledge 


Acquisition of knowledge depends upon learning, which requires both understanding 
and remembering. Empirical research, summarized by Simon (1979), has demonstrated 
that: 


4 skill gained in solving one form of a problem will not always transfer to 
isomorphs of the same problem (i.e., essentially similar systems, such as 
sales/receivables/receipts, to be evaluated for different clients may not 
benefit from skill transference). In general, skills are more likely to 
transfer downstream (i.e., from a harder to an easier task) rather than 
upstream (i.e., from an easier to a harder task); 


e rather minor and seemingly innocent differences in the "cover story" (or 
problem representation) can alter the amount of time required to perform 
a task; 

e changes in problem difficulty are associated with changes in problem 


representation; and, 


‘ no significant learning takes place in tasks requiring less than one hour. 
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In addition, as was pointed out in the discussion of expert knowledge, the weaker 
structural interweaving of patterns reduces the amount of learning that novices 
can derive from a given situation despite what may seem to an expert to be an 


abundance of information in the case. 
A Model of Professional Problem-solving 


By way of summarizing the foregoing discussion, a professional problem-solving 
model, based on a synthesis of the research relevant to the accounting professional's 


task environment, may be viewed as consisting of the following phases: 


a goal selection/clarification/definition; 


© observation of environmental cues; i.e., search, selection, screening, 
rejection and grouping of data; 


® application of specialized knowledge; hypothesis generation and cue 
interpretation; 
€ evaluation of problem representations and conditional search continuation 


(i.e., formulation of conclusions and strategies for further action, 
conditional on goal attainment); 


® learning; i.e., memory revision. 


In this model, (refer to Figure 2) the task environment is considered to be an 
objective cue-generator (i.e., the cues do not depend upon the problem-solver). A 
goal-directed, but otherwise relatively mechanistic, "operating system" senses 

cues, retrieves information from long-term memory, and deposits information into 
short-term memory. The representation in short-term memory of what is consciously 
known or understood about the problem under consideration includes the information 
so far gathered from the environment, active hypotheses (i.e., current speculations 
about what the problem is), and eventually, conclusions based on data. The generation 
and evaluation of hypotheses is affected by cognitive limitations, heuristics, 


judgmental biases and conscious strategies for acquiring and evaluating information. 


Johnson et al. (1981) emphasize the central role of knowledge in professional problem- 


solving. The expert auditor has: 
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MODEL OF PROFESSIONAL PROBLEM-SOLVING 


TASK ENVIRONMENT 


ENVIRONMENTAL 
CONSTRAINTS 


ENVIRONMENTAL 
CONSTRAINTS 


EXPERTISE 
LEVEL 


EXPERTISE 
LEVEL 


COGNITIVE 
LIMITATIONS ~* 


COGNITIVE 
LIMITATIONS 


PROBLEM PROBLEM REVIEWER 


SOLUTION 


FEEDBACK 


CUE CUE 


Figure 2 
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@ a collection of prototypes organized into categories and sub-categories 
around which facts would be clustered; 


® a list of procedures which should be performed when gathering task- 
specific information and knowledge of how such procedures should be 
executed; 

& knowledge of laws and principles governing the domain of his/her 
situations. 


Other personal attributes are excluded from the model (e.g., emotions, motives, 
personality traits, etc.) !® However, as Elstein et al. (1978) emphasize, the problem- 
solving process cannot be considered solely as a template-matching process. The 
problem-solver is not indifferent to the outcome to be selected. Thus, personal 
goals, values and other interferences will affect the outcome ultimately chosen. 

The greater the opportunities for applying discretion, the more likely it is that 
these interferences will influence the problem-solving process. As a result, to the 
extent that discretion is a characteristic of a professional practice, the judgments 
made will reflect the preferences of individual problem-solvers. This in turn leads 
to essentially non-comparable judgments, and the definition of expertise in such a 


case becomes quite tenuous. 


The greater the problem-solving skill (expertise) possessed by a professional, the 
fewer errors expected, relative to professionals with less expertise carrying out the 
same task. Indeed, even when they err, the errors of expert professionals are 
considered to be of a qualitatively different type than those of novices or laypersons 
(Johnson et al. 1981). However, as tasks become more complex, experts will likely 
commit more errors relative to their own performance in less complex tasks. Table | 
summarizes some of the potential audit consequences of human problem-solving 
limitations in the absence of compensating features. The precise nature of errors 
committed, including their frequency and pervasiveness among various tasks and 
various experience levels, are researchable issues with important implications for 


professional practice. 


16 Refer to Sjoberg (1982) for a discussion of these factors. 
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Table | 


What can go Wrong in the Absence of Compensating Features 


Flow of Audit Work through Time 


(Top to Bottom and Left to Right) 


PROBLEM- Documentation/ 
SOLVING Preparation of 

PHASE System Representations 
GOAL - Document the "wrong" 
PERCEPTION system 


- Stop too soon » 


CUE PERCEPTION/ - Gather insufficient 
RECOGNITION information 


Gather irrelevant 


information 
INFORMATION - Draw incorrect 
EVALUATION/ inferences about 
JUDGMENT the quality of the 

representation of 

the system 
CHOICE/ - Settle on an 
DECISION incomplete, confused, 


inaccurate, misleading 
portrayal of the 
system 


Source: Boritz (1981) 


Review/ 
Evaluation of 
Representations 


By Documentor 


- Misinterpret 
objectives 


- Answer the 
"wrong" question 


- Fail to see 
relationships 


- Focus on wrong, 
irrelevant cues 


- Draw incorrect 
inferences about 
the quality of 
system 


- Choose wrong 
aspects of 
system to focus 
upon as 
strengths and 
weaknesses 


Review Feedback 


by Reviewer | 


- Perceive goals 


incorrectly 


Inefficient search 
and restructuring 
of poorly 
structured 
representations 


Focus on wrong or 
irrelevant cues 


Fail to catch 


missing cues 


Draw erroneous 
inferences due to 
missing or 
incorrect informa- 
tion about both 
representation and 
system 


Choose wrong 
aspects to provide 
feedback 


Choose wrong 
corrective 
strategy 
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Ultimately, research efforts in these areas may permit the design of effective 
techniques for enhancing professional problem-solving skills. However, care must 


be taken to avoid the criticism of Simon (1965): 


All these aids to human thinking, and many others, were devised without 
understanding the process they aided -- the thought itself". (Simon 
1965, p. 92) 


It is of critical importance, prior to developing and testing problem-solving aids 

and judgment-enhancing techniques, that adequate research be conducted to provide 
an understanding of what goes on during the problem-solving process when 
accountants/auditors exercise their professional judgment, and in particular, the 
recurring kinds of errors or flaws that might to be prevented or minimized through 


the implementation of such systems. 
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SECTION TWO: PRINCIPLES OF JUDGMENT ENHANCEMENT OF RELEVANCE 
TO AUDITORS 


The problem-solving literature indicates that there are three major sources of 


errors: 


° errors due to reliance on heuristics (i.e., rules-of-thumb, short-cuts, 
etc.) which limit (a) the amount of information sought out and processed 
at any point in time, and (b) the sophistication of the combinatorial 
processes used for integrating various pieces of information; 


e errors due to the nature of the encoding, storage and retrieval mechanisms 
applied during the acquisition and use of professional knowledge, which 
limit the kind of information available for and used in problem-solving; 
and, ; 


e errors due to the nature of the task environment, in particular those 
aspects of complex tasks which prevent the problem-solver from gaining 
an adequate understanding of the essence of the problem. 


Various aids have been proposed to compensate for problem-solving flaws and to 
enhance problem-solving activities through the use of structure, mathematical 
models and automation. Slovic (1982) concludes that in developing aids to improve 
decision-making it is important to develop techniques for structuring decision 
problems and for simplifying the large complex decision trees which are 


characteristic of such problems: 


Now that we understand many of the biases to which judgments are 
susceptible, we need to develop debiasing techniques to minimize their 
destructive effects.... Simply warning a judge about bias may prove 
ineffective. Like perceptual illusions, many biases do not disappear 

upon being identified. It may be necessary to (a) restructure the judgment 
task in ways that circumvent the bias, (b) use several different methods 
allowing opposing biases to cancel one another, or (c) correct the 
judgments externally, based on an estimate of the direction and strength 
of the bias. 


Dealing with Biases 


Fischoff (1982) identifies a variety of strategies for dealing with biases (refer to 
Table 2) which depend on critical assumptions about the source of the observed 
behaviour such as whether the bias is the result of faulty tasks, faulty problem- 


solvers or a mismatch between problem-solvers and tasks. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 6, Section 2 


Debiasing Methods according to Underlying Assumptions 


ASSUMPTION 


Faulty tasks 


Unfair tasks 


Misunderstood tasks 


Faulty problem-solvers 


Perfectible individuals 


Incorrigible individuals 


Mismatch between problem-solvers 


and tasks 


Restructuring 


Education 


Source: adapted from Fischoff (1982) 


STLRATEGIES 


Raise stakes 

Clarify instructions/stimuli 
Discourage second-guessing 
Use better response modes 

Ask fewer questions 


Demonstrate alternative goal 
Demonstrate semantic disagreement 
Demonstrate impossibility of task 
Demonstrate overlooked distinction 


Warn of problem 

Describe problem 

Provide personalized feedback 
Train extensively 


Replace them 
Recalibrate their responses 
Plan on error 


Make knowledge explicit 

Search for discrepant information 
Decompose problem 

Consider alternative situations 
Offer alternative formulations 


Rely on substantive experts 
Educate from childhood 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 6, Section 2 - 464 - 


Faulty Tasks 


Not all flawed behaviour is attributable to limitations of human cognitive processes. 
Faulty tasks are tasks in which poor performance results from the nature of the 
task rather than the nature of the problem-solver. For example, a task may be 


considered to be faulty if a problem-solver: 


& doesn't care about it, 
& is confused by it or misunderstands its requirements, 
doesn't believe in the stated goal of the superordinate who designed the 


task or adopts other goals, 


oa can't express his or her true understanding due to limitations in the 
communication channel or inability to articulate inner thoughts, 


r falls into a stereotypic behaviour pattern simply to get through the 
task. 


Faulty Problem-solvers 


If a task is not faulty, then attention is shifted to the characteristics of problem- 


solvers instead, the subject of Section One of this chapter. 


There is a distinction to be drawn between perfectible problem-solvers and 


incorrigible problem-solvers. Perfectible problem-solvers can be helped through: 


warnings about the potential for bias, 
descriptions of directions of biases, 
feedback, 


training. 


Research suggests that the assumption of perfectible problem-solvers leads to 
unrealistic expectations and inappropriate blame and is of limited value in dealing 


with structural features of cognition. Incorrigible problem-solvers require continuous 
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support to achieve quality performance.!” Such support may be provided in the 
form of various debiasing techniques; for example, Mock and Vertinsky (1985) suggest 


the following techniques: 


& automatic correction for specific biases, 

® alert users about possible biases, 

® "triangulation" through multiple methods of eliciting judgments, 

e provide system-produced information prior to judgment (e.g., base rates), 
@ provide dialectical estimates (e.g., rival judgments, possibly random). 


Some or all of these strategies may be incorporated into systems designed to support 


managerial and professional problem-solving activities. 
Mismatch between Problem-solver and Task 


In many situations, it will likely be the case that performance quality will be 
contingent upon both the problem-solver and the task. Two main decision-support 


approaches are to: 


® restructure tasks to suit the individual, 


@ educate or train individuals to perform the tasks. 
Restructuring involves techniques aimed at: 


® encouraging/forcing problem-solvers to express what they know explicitly 
rather than simply accepting gut feeling intuition as the basis for 
judgment, 


a encouraging problem-solvers to actively search for disconfirming evidence 
rather than confirming evidence to corroborate a given (preferred) 
assertion, 


—_————_—. 


17. As used here, the term incorrigible is used to contrast with perfectible. No 
derogatory connotation is intended. 
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8 decomposing complex problems into several less complex components, 
* using different ways of looking at things, different terms, etc. 


Education, the alternative, involves the use of specialists or experts to replace 
faulty problem-solvers in performing the tasks. In some cases, these experts could 


be computer programs which mimic the problem-solving behaviour of human experts. 
Planning Aids 


Planning involves several important judgment activities, the most prominent of 


these being: 


@ problem definition and goal formulation, 
3 setting or identifying the problem-solver's expectations for outcomes 


(this sometimes includes estimating prior probabilities). 


Since problem-solving behaviour is so strongly goal-directed, it becomes important 
to use techniques which help the problem-solver identify and express the most 
appropriate definition of goals based on consideration of sufficient alternatives and 


focusing on the key information. 


In general, this may be accomplished by ensuring that: 


€ information is gathered from as broad a base as possible as part of the 
problem-definition/goal formulation activity. Also, new information 
should be added as soon as it becomes available, 

8 formal and disclosed methods are used rather than informal, hidden 
ones; i.e., assumptions are revealed and peer review is solicited. 


Techniques which may be useful for gathering and summarizing information for 


planning purposes are: 


® Delphi, 
a Analytic Hierarchy Process, 


& "Policy Capturing" (sometimes called bootstrapping), 
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w Simulation. 


These are described below at a summary level to introduce the techniques. More 


detailed descriptions may be found in the references cited. 


Delphi 


Delphi is a group-process technique for eliciting, collating and directing informed 
(expert) judgment toward a consensus on a particular topic (Delp et al. 1977, p. 68). 
Individuals debate anonymously by mail through a set of questionnaires. The 
responses are then collected, collated and analyzed by a design team. Based on the 
results of the analysis, another questionnaire is developed. The process continues 
with rounds of questionnaires until all opinions converge. This technique is useful 

in soliciting opinions from various groups in the organization in the process of setting 
audit objectives, identifying important concerns, creating a system of priorities 


and designing an audit framework. 


The Delphi technique can assist in establishing goals and priorities which are 
acceptable to the entire organization, thus contributing to congruence between a 
department and the larger organization. It can be used to define the scope, 
dimension and attributes of the problem, and permits equal participation of groups 


with varying backgrounds. 


The technique allows active participation of well-informed executives who are 
geographically scattered and also allows active participation of knowledgeable 


executives who cannot afford the time required for group meetings. 


The anonymity provided by Delphi may reduce the impact of biased opinions of 
certain dominant groups and may elicit more genuine responses than might otherwise 
be obtained; for example, in group meetings. By having individuals think through 
and respond on their own, the Delphi process avoids tunnel vision and bandwagon 


thinking and encourages diverse and speculative thinking (Wedley 1977). 
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Ratings from Delphi studies provide quantitative scores for evaluations even though 
these evaluations involve subjective and intuitive thinking. These ratings can aid in 
choosing a course of action (Wedley 1977). The number of Delphi respondents can 


be incremented with very little extra cost. 


One of the potential limitations of Delphi is that desirable features of a group 
meeting, such as instant communication, brainstorming and intellectual stimulation, 
may be lost (Delp et al. 1977). In addition, it requires some time, approximately 

six weeks, to gather, assimilate and analyze responses. The respondents must have 
good written communication skills, as well as a high degree of interest and 
commitment throughout the process to maintain the necessary quantity and quality 


of responses after successive rounds of questionnaires. 


Delphi can help the internal audit department gather informed outside opinions as a 
basis for organizing and directing its activities. This planning exercise cannot be 
carried out frequently due to the effort required on the part of the participants as 
well as the coordination required while it is being carried out. It can be applied 

once every three to five years and in a very specific way help identify the key 

factors to be used in important planning decisions such as audit-priority determination, 
personnel arrangement, coordination of plans with external auditors and auditees, 


areas of emphasis, etc. Further details may be found in Boritz (1983b). 


Analytic Hierarchy Process 


Saaty (1980) described analytic hierarchy process (AHP) as "decomposition by 
hierarchies and synthesis by finding relations through informed judgment". The 
idea is that a system is better perceived by decomposing the complexity of the 
structure into its components and finding hierarchical relationships among them. 
The relative strength with which elements at one level influence those of the upper 
level is measured by a series of pairwise comparisons (i.e., evaluating components 


two ata time). 


The concept of AHP may be useful in structuring audit-risk evaluation and personnel- 
development decisions, among other analyses. Figure 3 illustrates the hierarchical 


relationships among overall audit objectives, risk factors and audit units. The 


Internal Audit Handbook 
Volume II, Part 2 


Chapter 6, Section 2 - 469 - 


ANALYTIC HIERARCHY PROCESS 
Source: Adapted from Patton et al. (1982) 


Figure 3.A 
Analytical Hierarchy 


Level One: Overall Objective 


Risk 


Level Two: Criteria Factor 


Risk Risk 
Factor Factor 
1 2 


Level Three: Area of Interest 


Source: Patton group (adapted). 


Figure 3 
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Figure 3.B 


Risk-Factor-Importance Scale 


Comparison Matrix 


1 2 

1 5 
1/5 1 
1/3 2 
1/2 3 
1/7 1/2 


3 


4 5 
2 7 
WA 72 
1/2 3 
1 4 
1/4 1 


*The eigenvector has been normalized to sum to one. 


Audit Unit 1 
Audit Unit 2 
Audit Unit 3 
Audit Unit 4 


Figure 3.C 


Audit-Unit-Risk Scale 


Size Factor* 


AU2 


=—nrna 


Scale 


Eigenvector* 


AU4 Scale 


5735 

2334 
-1287 

-0644 
1.0000 


— bo & OO 


*The lower half of the matrix is not filled in because, under this methodology, 
these values are constrained to be reciprocals of the corresponding entries in 
the upper half; e.g., the entry for the fourth row and first column would be 
“‘forced’’ to be 1/8. 


Size 


(from Figures 3.B, 3.C) 


AUT 5735. 
AU2 _ .2334 
AU3 _ .1287 
AU4  .0644 

1.0000 


*This is the normalized result of multiplying (i.e., by matrix 


Audit-Unit-Risk Matrix 


Liquidity Personnel 


-1167 
2107 
-0770 
5956 
1.0000 


System Complexity 
(figures arbitrarily selected) 


-1602 
2726 
-0999 
-4673 
1.0000 


-1096 
-2672 
-0560 
-5672 
1.0000 


Overall 
Risk 
Measure* 


0.4002 
0.2343 
0.1238 
0.2417 
1.0000 


multiplication) the eigenvector in Figure 3.B and the Audit-Unit-Risk 


Matrix in Figu 


re 3.C. 
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apex shows an example of the overall objective which an organization might aim to 
achieve. To achieve this objective, individual risk factors have to be identified and 
eliminated or minimized. To minimize risk, audit resources have to be allocated 
among audit units depending upon their individual risk exposure, assuming that 


audits reduce risk. 


The relative importance and extent of risks of each audit unit can be measured by 
pairwise comparisons based upon the professional judgment of audit planners and 
recorded in a table. Through the application of mathematical techniques to the 
table of pairwise comparisons, the underlying scale values can be derived. The 
overall risk for each audit unit is determined by calculating a weighted average- 


risk measure. 


AHP provides a framework which aids in the systematic and scientific evaluation 
of risks affecting audit units and a logical link between the amount of risk and 
resource allocation. An open-group process is used for determining risk factors 


which may encourage intellectual stimulation. !8 


A higher level of confidence is 
placed on the relevant risk factors since they are agreed upon by the group, and the 


derived scale values represent the most consistent judgment of the group. 


Some limitations of AHP are that the derivation of the scale values from the 
pairwise-comparison matrix is a complicated mathematical process and requires 
computer software. Also the process becomes time-consuming as the number of 


comparisons increases. 


Additional information may be found in Boritz (1983b) and Patton et al. (1982). 
Lin, Mock and Wright (1984) describe the use of AHP as an aid in planning the nature 


and extent of audit procedures. 


18 But this group process may also have undesirable consequences, as discussed 
at the end of Section Two in "Humans as Synthesizers". 
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Both Delphi and AHP are useful techniques in structuring risk evaluation. !? Both 
approaches aim at reaching consensus through simple, repetitive processes. Delphi 


uses rounds of questionnaires, and AHP uses a series of pairwise comparisons. 


With Delphi, participants respond anonymously to avoid the influence of a dominant 
group, whereas with AHP, an important set of variables and judgments is established 
by an open-group process. In Delphi, disagreements are resolved by successive 
review and revision of questionnaire results on an anonymous basis, whereas in 

AHP, differences in views are resolved by open discussion among informed 


participants. 


Delphi supports statistical and quantitative analysis of numerical responses gathered 
from questionnaires, whereas AHP uses absolute numbers, one to nine, to reflect 
qualitative judgments based on pairwise comparisons that are subsequently used to 
derive an estimate for the underlying scale. Lastly, Delphi has no built-in mechanism 
for checking the consistency of responses, whereas AHP permits consistency 


checking. 


Policy Capturing (Bootstrapping) 


An early demonstration of bootstrapping by Yntema and Torgersen (1961) suggested 
that if an automated aid could be developed to capture the judgments or policy of 
an expert, then its performance will be better than (or at minimum, as good as) the 
expert's unaided judgment process by virtue of eliminating inconsistencies in human 
task performance from case-to-case or time-to-time. Camerer (1981, p. 411) 
concluded that, "bootstrapping will improve judgments slightly under almost any 


realistic task condition". 


19 There are other approaches (e.g , multi-attribute utility decomposition 
(MAUD)) designed to help decision makers in structuring, decomposing and 
recomposing preferences in situations involving several alternatives which 
have several dimensions (attributes) of importance to the decision. The value 
of these approaches is through reduction of goal confusion and through 
consciousness raising about the structure of attribute importance ratings 
(Humphreys and McFadden 1980). For further information refer to Edwards 
and Newman (1982). 
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Bootstrapping is based on the use of linear statistical models such as regression 
analysis. The models are developed by using data gathered from repeated applications 
of judgment, either for one person over time, or for many people at one point in 


time. 


Regression analysis is a technique that expresses the average relationship between 
two or more variables. It relates a dependent variable with an independent 
variable(s) in the form of a mathematical equation derived by the "least-squares" 
method. It is a relatively straightforward process for establishing a norm for a 
given relationship based on the patterns of past data. However, it may be misused 
if the relationship between the dependent variable and independent variable is valid 
only within a restricted range. Also the regression model may be invalidated when 
factors that govern the relationships are changed and require a new model. In 
addition, aggregation across people or time periods may not always be valid. 
Although it requires some tedious computation, computer programs are available 


for this purpose. 


One of the fundamentals in this technique is that the model captures the important 
factors and eliminates (i.e., relegates to random error) the unsystematic interfe- 
rences due to heuristics and biases. Figure 4 illustrates this technique; further 


information about this technique may be found in Einhorn (1972). 
Simulation 


Simulation is a process of conducting experiments on a model of a dynamic system 
in lieu of either direct experimentation with the system itself or direct analytical 
solutions of problems associated with the system. It is a symbolic or numerical 
abstraction of the process under study and not the process itself. For example, 
audit plans are vulnerable to changes in business operations such as acquisitions of 
new subsidiaries, automation, budget cuts, work force shortages, changes in key 
audit personnel, changes in risk factors and similar events. By changing relevant 
parameters to correspond with expected changes in relevant factors, possible effects 


of changes on audit plans can be observed and studied. 
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EXAMPLE OF REGRESSION COMPUTATION FOR 'BOOTSTRAPPING' 


Past Data 
Audit period 1 2 3 4 
Estimated complexity points (x) 350 300 150 250 
Audit time ( hours, y) 1000 950 500 750 


Computation of Regression Coefficients 


(5) 
(x-x) (y-¥) 


21,000 
300 8,000 
150 - - 29,000 
250 


- 0 
200 - 


20,000 


Total 1,250 78,000 


78,000 
b= —— =3.12 
25,000 
b=3:12 
a = 790 - (3.12)(250) 
= 790-780 
= 10 
Regression equation is y = 10 + 3.12x 


> ¢ 
y = 10+ 3.12x 
Hours 
(in hundreds) 
20 (500,1560) 
OR 2 3) 4) Ome 8 1 9) 0 
Complexity points (in hundreds) 


Figure 4 
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Simulation is the most effective means of analyzing complex systems when analytic 
and numeric solution methods are deficient for unravelling a problem or are 
impractical to apply. The approach shows the effects on system components of 
varying conditions, assists in forecasting system behaviour, and permits study of a 


wide range of alternative policies without actually working on the physical system. 


Potential limitations of a simulation model are that it does not by itself provide a 
solution to any problem, but provides an understanding of the relationship among 
the components of a system. The usefulness and reliability of conclusions derived 
depend on how closely the model represents reality. To achieve this, simulation 
experiments may require large volumes of data. The unavailability of adequate 
data is often a major obstacle in developing simulation models. Rigorous application 
of this technique requires familiarity with simulation statistics, computers and 
interactive terminals; thus, a payoff may come only after auditors gain necessary 
experience. This potentially learning curve may be costly and time-consuming. 
Less rigorous methods may, of course, be used; e.g., the widespread use of spread- 
sheets for "what if" types of analyses is a less rigorous, yet still useful, method for 


simulating the behaviour of quantifiable variables. 
Information Gathering Aids 


Elstein et al. (1978) studied doctors' diagnosing behaviour. They found that every 
physician who at any time considered the correct solution to the diagnostic problem, 
selected that solution only as the final diagnosis. This finding serves to emphasize 
the importance of preventing premature restriction of hypotheses entertained by 
professional problem-solvers. There are two key aspects to ensuring adequacy of 


information search: breadth of search and depth of search. 
According to Cutler: 


Many educators and physicians regard all of diagnosis as a huge branching 
tree. They feel that diagnosis consists of simply tracing one of a large 
number of pathways through a system of branches. Each of these systems 
is in reality an algorithm or series of questions with yes or no answers 
directing the branching process. The physician quickly derives a few 
important clues and departs down one branch of the diagnostic tree. If 
pursuit is hot, questions continue along that line. But if he finds by a 
series of negative responses that he is going in the wrong direction, he 
comes back to a node and follows another branch. (p. 38) 
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Breadth of search requires searching across a sufficient number of nodes at a given 
level of a hierarchy. Depth of search requires searching down through a sufficient 
number of levels of the hierarchy. Although the analogy between medical and 
audit information-gathering should not be stretched too far, the hierarchical, 
structured approach to information gathering described can be helpful in ensuring 
complete and thorough search by a problem-solver, possibly utilizing expert guidance 
embedded or incorporated in the search tree.2? Second, information gathered in 
this way can be readily transferred to graphical representations such as flowcharts 
and/or other diagrams. Third, the process can be computerized and guided by an 
unobtrusive operating system free of biases, which could, in addition, combine 
cues and generate hypotheses for consideration by the auditor. These are the tasks 


which, as we have previously noted, professionals do not perform particularly well. 


In a similar vein (see Figure 5) Fischoff et al. (1978) describe the fault tree 
technique and its application to such complex systems as nuclear power plants. 
Libby (1981) suggests that fault trees could be used as extensions of flowcharting 
techniques since they, in contrast with the flowcharts' emphasis on transaction 
flows, emphasize possible system failures. When fault trees are used in conjunction 
with analyses of complex systems, the approach is to first define sub-system failures 
which would cause overall system failure. Then, individual component failures 
related to each sub-system failure are listed (as branches) along with their inter- 


relationships. 


The construction of fault trees forces pre-specification of all conceivable causes of 
a given failure. Thus information search and hypothesis generation are guided and 
the problems associated with premature hypothesis restriction can be avoided or 
reduced. It should be noted that preparing such specifications is not an unreasonable 
effort to be expended since, although in a more haphazard and often implicit fashion, 
such specifications form part of most large public accounting firms' audit programs. 
One of the strengths of Computer Control Guidelines (CICA 1970) was that it took 


essentially this approach (refer to Figure 6). 


20 Dickhaut and Eggleton (1975) used this technique to elicit the heuristics their 
subjects used during an experimental task. 
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FAULT TREE 


Source: Slovic, P., Lichtenstein, S., and Fischoff, B., "Images of Disaster: 
Perception and Acceptance of Risks from Nuclear Power", in G. Goodman 
and W. Rowe (eds.) Energy Risk Management, London: Academic Press, 
Be he 
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FAULT TREE STRUCTURE OF CICA COMPUTER CONTROL GUIDELINES 
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Figure 6 
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Documentation Aids 


Although many forms of audit documentation are in use, they have often been 
considered as substitutes for one another, rather than as specific tools useful for 
specific aspects or phases of the auditor's problem-solving activities, and less useful, 
even dysfunctional, for other aspects. Whereas in well-structured problems such 
substitutability may be possible, it is unlikely to be very effective in complex task 
environments. Nor can one method serve all phases of problem-solving. Some 
techniques may enhance the information search process, whereas different techniques 
may be particularly valuable for enhancing evaluation phases of prablem-solving by 


auditors. 


In developing the technology of auditing, it seems appropriate to take into account 
the purpose and potential usefulness of various audit documentation techniques, 

and with these in mind, to tailor existing techniques, or develop new ones to enhance 
audit problem-solving. Such investment in technological improvements in auditing 
is particularly feasible since many audit tasks are repetitive and since there exists 


a relatively well-specified procedural knowledge base. 
Simon (1969, p. 109) reminds us: 


One might suppose that the description of a complex system would itself be a 
complex structure of symbols... but there is no conservation law that requires 
that the description be as cumbersome as the object described. (Simon 1969, 

p. 109) 


Complex systems are hierarchical in nature and tend to display a high degree of 


cue redundancy. As such, Simon (1969, p. 110) makes the following assertions: 


e They are usually composed of only a few different kinds of sub-systems, 
in various combinations and arrangements, so that hierarchic represen- 
tation can be a major facilitating factor for simplifying and thus 
enhancing both the description and understanding of complex systems. 


® They are often nearly decomposable; i.e., only aggregative properties of 
their parts enter into the description of the interactions of those parts. 
Thus little information is lost by using hierarchical representation, 
providing that such representations are prepared by trained individuals. 
Otherwise, if untrained individuals do prepare the representations, 
detailed information about relations of sub-parts belonging to different 
_ parts is likely to be lost. 
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® By appropriate "recoding", the redundancy that is present, but unobvious, 
in the structure of a complex system, can often be revealed; for example, 
replacing a description of a sequence of steps by a description of the 
process that generates that sequence, can help simplify an otherwise 
complex problem. 


Alexander's (1964) classic work devoted considerable attention to the problems of 
representation. In his view, diagrams may have two important and distinct qualities 
(p. 89): 


% they may summarize "physical" structure (i.e., the formal characteristics 
of what the system is), and 


8 they may summarize a set of functional properties or constraints (i.e., 
denote the goal to be achieved, or what the system does). 


Moreover, Alexander asserts that good diagrams contribute to understanding not 
just the specific problem at hand, but also the broader concepts reflected therein. 
Since we seldom understand the broader conceptual context fully, we may not at 
the outset see it as a single pattern; therefore, diagrams often precede the precise 
knowledge which, if it existed, could prescribe their shape on rational grounds. 
However, some principles can help guide the representational direction that ought 


to be taken. According to Alexander (1964, p. 127), each diagram must: 


@ bring out just those features of the problem which are relevant to the 
given set of requirements, 


& include no information which is not explicitly called for by the 
requirements, 
% be so specific that it has all the physical characteristics called for by 


the requirements, yet 


@ be so general that it contains no arbitrary characteristics, and so 
summarizes abstractly, the nature of every representation that might 
be satisfactory. 


Although the complexity of internal interactions may make it impossible to find a 
single adequate diagram, simpler diagrams may help to get at the problem, and 


Alexander concludes that: 
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® A hierarchical representation of the most significant sub-sets of the set 
of requirements may help find such simpler diagrams. 


@ Each sub-set can in turn be translated into a set of smaller sub-sets 
which, by virtue of having fewer requirements and interactions, are 
able to be better understood. 


These points, however, are easier said than done, since in fact there are two opposing 
goals -- analysis (decomposition) and synthesis. The initial goal is to fragment the 


problem into smaller sub-problems. 


The ultimate goal, however, is to form a unit, bringing elements together in such a 
way as to create one cohesive solution. The need for sub-sets which can be grasped 
through the use of diagrams calls for sets of variables whose internal interactions 
are very rich, or tightly coupled functionally. However, the need to resolve conflicts 
between sub-sets calls for as little interaction between them as possible. Therefore, 
to accomplish these goals, we should begin by constructing diagrams for the smallest 
sets prescribed, building up compound diagrams according to the hierarchical 
structure used to sub-divide the larger problem in the first place. In other words, 
this means top-down sub-division (i.e., division and partitioning) of a system into a 
tree of components, but bottom-up representation (i.e., successive composition and 


fusion) to obtain a tree of diagrams.7! 


Thus, the tree becomes the theoretical basis for all representations. A tree is a set 
of sets of variables whose relevance can only be understood in terms of their 
functional relationship to the other variables in the set. With reference to Figure 7, 
compare panels X and Y. It seems apparent and obvious that Y is far more straight- 
forward in appearance than X; however, X seems to be a closer approximation of 
what we are accustomed to seeing as systems descriptions today. In advancing the 
tree (hierarchy) for representation of complex processes, Alexander (1964) asserts 
that it provides an explicit description of the implied structure of a process, and it 
gives the strongest possible decomposition of the problem that does not interfere 
with the task of synthesizing its parts in a unified way (i.e., each subsidiary problem 
defined has its own integrity and is as independent as it can be of the rest of the 


problem). 


21 ‘For an interesting application of these concepts refer to Pirsig's, Zen and the 
Art of Motorcycle Maintenance (1974). 
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HIERARCHICAL REPRESENTATION 


Figure 7 
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Structured Representation Techniques 


Even a casual review of some of the structured systems analysis and design 
methodologies reveals that they are, virtually without exception, essentially 
hierarchical decomposition and representation "technologies" grounded in the 
preceding theoretical arguments. There are a number of techniques, each with 
supporters and detractors; while comparison of their merits and deficiencies is 
outside the scope of this chapter, a good comparative analysis can be found in 


Peters (1981), who classifies representation techniques into four categories: 


system architecture, 
design structure, 


database structure, and 


software behaviour. 


Representing system architecture: Peters (1981, p. 43) claims that system 
architecture is the least understood and most poorly addressed area of representation. 


Its objective is to help in the communication of primary conceptual issues by 
depicting major portions or functions of a system and their relationship to one 
another. Therefore, this type of representation must highlight the span of control 
and the relative importance of individual sub-systems in the larger scheme of the 
system. In addition, representation techniques in this category must provide a 


means of focusing upon the critical issues or the goals set for the system. 


22 Although both Simon (1969) and Alexander (1964) conclude that there are 
essentially two types of representations for describing complex systems (i.e., 
state/structure descriptions, and process/ function descriptions), Saarinen 
(cited in Peters 1981) adds a third factor, style -- i.e., why a system does 
what it does in a particular way. Peters (1981) building on these three elements, 
function, structure and style, identifies four categories of representation 
techniques; i.e., system architecture, design structure, database structure 
and software behaviour; and then classifies twenty-eight representation 
techniques proposed for describing computer-based systems (unfortunately, 
the link between representations of manual portions and computerized portions 
of systems is not made clear). 
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In particular the following are considered to be the important attributes in 


representations of system architecture: 


- hierarchic organization of system elements, 

- sequence (precedence) of processes, 

- priority levels of processes, 

- external interface with parties outside the system boundaries, 

- interactions among modules within the given application system, 

- internal interfaces between application system programs and the 
operating system, 

- user requirements expected to be met by system functions, 


- process logic of modules within the application system. 


Peters rates Leighton diagrams as being currently the most effective technique for 
representing system architecture. An example of a Leighton diagram is provided in 
Figure 8. This method is used to represent software system architecture in an 
hierarchical manner, but avoids detail in favour of a highly simplified, easy-to- 
understand format. The ABC Sales System presented consists of five main modules, 
whose sequence of execution (in general) flows as indicated from top to bottom. 
Each module in turn consists of sub-modules, the "calling" hierarchy moving from 
left to right. Finally, the inputs and outputs are indicated at the extreme right, 
with source/destination device and nature being shown by using standard flowchart 
device symbols and vectors. Boritz (1983a) applied Leighton diagrams to internal 


control questionnaires (see Figure 9). 


Representing design structure: Structure is an umbrella term for the many types 
of relationships present in a given system; e.g., hierarchy, inclusions, equivalence, 
precedence, execution sequence, and scope of control. The important attributes of 


representations of design structure include the following: 


- hierarchical organization of system elements including separate and 


specific treatment of data flow and control flow, 


- capability for portraying precedence or sequence of processing events, 
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REPRESENTING SYSTEMS ARCHITECTURE: LEIGHTON DIAGRAMS 
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USE OF LEIGHTON DIAGRAM TO HIGHLIGHT STRUCTURE OF INTERNAL CONTROLS 


Source: Boritz (1985a) 


ORDER 
INITIATION 
ACCEPTANCE 
PARTI 
AUTHORIZ- aes aa 
ATION 
OBJECTIVES 
ADJUSTMENTS 
ALL GOODS 
INVOICED 
COMPLETENESS 
OBJECTIVES ALL CASH 
RECEIVED 
RECORDED 
RECORDED 
VALID 
VALIDITY 
OBJECTIVES RECORDED 
RECEIPTS 
VALID 
PART Il Pannier DEBITS 
ACCOUNTING AMOUNTS CREDITS 
INTERNAL OBJECTIVES 
CONTROL DEBITS 
EVALUATION CORRECT 
GUIDE PERIOD CREDITS 


ACCURACY 
OBJECTIVES CORRECT SS FeEITS | 
ACCOUNTS GRR 


POSTINGS 


SHIPMENTS 


CASH 


PART Ill RECEIPTS 


ASSET FORMS 


SAFE- 
GUARDING 
OBJECTIVES 


RECORDS 


OPEN ITEMS 


Figure 9 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 6, Section 2 - 486 - 


- means for representing process logic of individual modules, 


- explicit depiction of individual modules and the communication among 


them. 


Structure charts were originally developed by Stevens, Myers and Constantine (1974). 
A hierarchical approach is used to represent module interaction, and information 
communicated between modules is explicitly shown, in particular data flows and 
control parameters are graphically depicted. Figure 10 shows the design structure 
of the order entry module previously shown in Figure 8. Modules are represented 

by rectangles. Data flows are represented by arrows with open circles, and control 


parameter flows are represented by arrows with closed (shaded) circles. 


Representing database structure: Although this is a newer area of representation 

than the activity-oriented schemes, it has grown in importance in recent years. 

Activity-related schemes are unable to highlight important logical attributes of 

data; and so, as data management has increased in importance, so to have database ¢ 
representation schemes. 7 


Auditors have, by and large, avoided using data analytic?? 


techniques. Indeed in 
the past it was both convenient and justifiable to focus attention on physical files 
of data as units of control, rather than selecting logical elements within those 
files, because the two were combined. This is no longer the case. However, 
continued use of physical files as units of control has left many procedures ill- 
specified in terms of the logical entities, their attributes and the relationships 


among them, to which audit attention should be given. 


For example, is the product master file the unit of interest, or is it specific 
attributes of products (e.g., the price and quantity fields) which are of audit 
importance? And if the latter, then the fields are irrelevant without the following 


relations being considered as well: 


23 Some of the techniques in use today include: Chen's entity-relationship 
approach (discussed in Peters, 1981, Ch. 5), Senko's (1975) logical data structure, ¢ 
data dictionary systems, data structure diagrams, problem analysis diagrams, 
and a number of other less well-known techniques described by Peters (1981). 
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REPRESENTING DESIGN STRUCTURE: STRUCTURE CHARTS 


ABC 
SALES 
SYSTEM 


ORDER 
ENTRY 
SHIPPING 
CREDIT OK’D INFO 
OK 
ORDER ] STOCK 


o> 


STATUS 


CREDIT STOCK PRINT 
CHECK CHECK SHIPPING 
DOCUMENT 
EDITED 
ORDER 
INFO 


GET EDITED 
ORDER 
INFO 
TRANSACTION TRANS MESSAGE 
INFO 
EOT 
INVALID 


READ EDIT WRITE 
TRANS TRANS MESSAGE 


Figure 10 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 6, Section 2 - 488 - 


- price-of-part-number, 


- quantity-of-part-number. 


Mair et al. (1978) and others have suggested that auditors' analyses be refined by 
focusing on specific data elements and logical relationships among them. Auditors 
should consider adopting data analytic techniques such as logical data structure 
diagrams as aids for separately representing logical and physical data structures, 
isolating key entities and relationships, evaluating controls and designing audit 


tests. 


An example of a logical data structure diagram is presented in Figure 11. Its 
essential features include the use of simple ovals to represent entities and attributes 
and named arcs to represent relationships. To represent many-to-one relationships, 
"chicken-feet" are added to the appropriate end of an arc. Many-to-many relation- 
ships are not permitted, except if redefined into their component many-to-one 


relationships. 


Representing software behaviour: Flowcharts have been traditionally one of the 
most popular audit documentation tools; however, they have fallen into disrepute 
among current writers in the software engineering field such as Yourdon and 
Constantine (1979). Peters (1981) asserts that flowcharts are based on the belief 
that a program should be documented after it is written. As a result, flowcharts 
suffer from several weaknesses which limit their usefulness. They are not considered 
very usable in the process of arriving at descriptions of system behaviour; but, as 
has been argued previously, a primary purpose of representation techniques should 
be to help auditors during their problem-solving activities. To be fair, flowcharts, 
prepared after the system behaviour has been determined by other means, do 
contribute to communication with reviewers and auditors in subsequent years; 
however, Peters (1981) and others as well, argue that there is little motivation for 
producing a complete and clear representation after the fact, making traditional 
flowcharts generally inaccurate and incomplete representations of system behaviour. 
To those who are accustomed to flowcharts, these criticisms may seem unfair. It 


may be that they do not apply to audit uses of flowcharts to the same degree as to 
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REPRESENTING DATABASE STRUCTURE: LOGICAL DATA STRUCTURES 


Source: March, S.T. and J.V. Carlis, "A Computer-Aided Database Design 
Methodology" Working Paper, Minneapolis, MN: MIS Research Centre, 
University of Minnesota, September 1980. 
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software development uses.2+ Programming theoreticians (e.g., Knuth, 1974) have 
shown that any program may be represented as an arrangement of the three simple 


structures, shown in Figure 12: 


- a sequence (i.e., do part 1, part 2, part 3... end do), 
- an alternation (i.e., if... then...else... end if), 


- an iteration (i.e., while. ..do...end while). 


This arrangement is consistent with the argument presented earlier regarding 
reduction of complexity by taking advantage of the redundancy present in complex 


systems. 


Some of the essential attributes of software behaviour representation schemes 


include: 


- ability to represent data flows, data structures, control flows and 
control structures, 


- forced simplification to the three basic constructs; i.e., sequence, 
iteration, alternation, 


- distinction between physical and logical form; i.e., language free, 


- explicit but clear depiction of nesting levels, highlighting the required 
conditions for a program segment to execute, 


- real time effects/responses represented (as opposed to batch). 


A number of techniques have been proposed for representing software behaviour, 
including: various forms of pseudocode, control graphs, decision tables, flowcharts 
and so on. Although several of these techniques might suggest useful adaptations 
for audit purposes in specific instances, no technique except perhaps pseudocode 


appears generally applicable. 


24 This is an empirically testable issue and requires some evidence to help resolve 
it. One thing is certain; the criticisms should not be rejected out-of-hand. 
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REPRESENTING SOFTWARE BEHAVIOUR: BASIC CONSTRUCTS 
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Information Evaluation Aids 


Slovic and Fischoff (1977) outlined a method designed to reduce hindsight bias in 
judgment formation. Under this method, auditors would be asked not only to explain 
why a given conclusion was drawn but also to explain the circumstances that would 
have led to an alternative conclusion; for example, not just how a control works but 


what circumstances would lead it to fail. 


Koriat et al. (1980) suggested that requiring two lists of reasons, one supporting a 
given conclusion, the other supporting the opposite, can produce a marked improvement 
in the appropriateness of the amount of assurance (or confidence) expressed about 


a conclusion. 


Boritz (1983a) adapted these suggestions. in designing an internal control 
questionnaire (see Figure 13). The parallel organization of "yes" and "no" answers 
to specific points is intended to encourage careful weighing of those responses and 
considerations prior to making a global "YES" or "NO" judgment about an overall 


control objective. 


Usually, after numerous items of information have been identified it becomes 
important to evaluate and combine them into a global judgment or decision. For 
this purpose there are essentially three categories of techniques which may be used 


to assist the problem-solver: 


z graphical or tabular data presentations, 
management science models for synthesizing or aggregating data, 
@ humans for synthesizing and aggregating data (including expert systems). 


Another possibility involves using a combination of these approaches. 
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EXCERPT FROM STRUCTURED INTERNAL CONTROL EVALUATION GUIDE 


Source: Boritz (1985a) 


CLIENT: Dominion Hardware 
Wholesalers Limited 
INTERNAL CONTROL EVALUATION GUIDE 
SALES, RECEIVABLES, RECEIPTS 
AUTHORIZATION OBJECTIVES 


Only sales that meet management's authorized criteria, such as 
customer acceptability, credit worthiness, prices, delivery terms, MES NO 
and other terms of sales are accepted. 


Established policies and well-defined procedures for n/a yes no 
investigating credit worthiness of prospective customers. 
Use of customer lists (or master files) approved by credit n/a yes no 


manager with adequate procedures for, and controls over, 
adding, changing, and deleting information from such lists (or 
master files) on a timely basis. 


Well-defined criteria, procedures and responsibilities for n/a yes no 
approval and processing of customer orders. 
Programmed procedures for checking validity of customers n/a yes no 


and terms prior to processing sales orders. 
Adequate controls over program development and modification. n/a yes no 


Sales prices, quantities and other terms conform to the 


authorization. MES NO 
Appropriate order entry forms (or video terminal screen n/a yes no 
layouts) used. 
Approved current sales catalogs or standard price lists used. n/a yes no 
Appropriate pre-programmed criteria used to adequately 
monitor and control order entry procedures. n/a yes no 
All departures from established criteria (e.g., standard prices, 
terms, credit limits) require special approval. n/a yes no 
All exceptions to pre-programmed criteria automatically n/a yes_—no 
logged and subsequently reviewed. 
Periodic review of credit limits and other standing customer- n/a yes no 
related information. 
Periodic independent check of sales order details against n/a yes no 


management's authorized criteria. 
Sales related deductions and adjustments such as credits for return 
goods, services under a warranty, cash discounts, and sales YES NO 
commissions are properly authorized. 


Properly approved commission schedules used. n/a yes no 
Adequate procedures for approval of bad debt write-offs. n/a yes no 
Adequate procedures for handling goods returned for creditor n/a yes no 
warranty service. 

Appropriate pre-programmed criteria used to adequately n/a yes no 
monitor and control entry of miscellaneous credit 

transactions. 


Figure 13 
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Graphical Data Presentations 


Recall that many of the judgmental biases discussed previously emphasized human 
weaknesses in properly treating multi-variate, covarying, cues. Cluster represen- 
tations are one way of compensating, by representing the associated variables 
graphically. Although such diagrams are more readily prepared, given today's 
technology, for quantitative data, the concept merits consideration for qualitative 


data as well. 


A particularly innovative technique for graphically representing multi-dimensional 
data (see Figure 14) was suggested by Chernoff (1973). Faces are constructed 
(manually or by computer) by assigning each variable of interest to a feature of a 
face (e.g., size of eyes, length of nose, etc.). Moriarity (1979) contended that 


schematic faces have particular potential because: 


e people are familiar with faces so that they can easily distinguish change, 

® the faces are rich enough in detail to represent a large number of 
variables, 

e there is a good deal of psychological research indicating the saliency of 


various facial features, and these may provide a basis for assigning 
weights to features, 


* technology for their construction is available and inexpensive. 


A major weakness of the method might be its susceptibility to being underestimated 
in terms of its usefulness and effectiveness because of the cartoon-like faces that 
result. Even of greater significance is the dependence of the face-drawing technology 
upon a statistical model of the phenomena to be represented. Such models do not 


exist for most problems that auditors deal with and would need to be constructed. 


Experienced or expert professionals often recognize clusters and patterns of cues 
without aids. These are part of their knowledge base. To the extent that cluster 
analysis and pattern recognition can be formalized, less expert auditors would benefit 


by improved ability to integrate cues or observations. 
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CHERNOFF FACES 


Source: Moriarity (1979). 
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Figure 14 
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Ehrenberg (1977), as cited by Libby (1981), suggests that a good table should make 
patterns and exceptions obvious when the probable pattern is known beforehand. 
For example one auditing firm pre-specifies "critical" combinations of controls 
(absence/presence) which are used as control evaluation aids in a micro-computer- 


based decision table methodology (Deloitte, Haskins and Sells 1985). 


Management Science Models 


Linear programming: Linear programming is a mathematical method of determining 
an optimal solution that satisfies multiple objectives and numerous interrelated 
restrictions and constraints. A linear programming problem has three essential 


elements: 


~ alternative courses of action - there must be two or more controllable 
variables that must be handled simultaneously. 


- constraints - the alternative courses of action or the variables of the 
model are interrelated through some type of restriction. Restrictions 
define the feasibility of a proposed course of action. 


- objectives - there must exist a clear-cut criterion by which the relative 
merits of each of the alternative courses of action may be evaluated. 


Linear programming may assist the auditor in ensuring that professional objectives 
are met subject to specified resource limitations. It allows flexibility by permitting 
numerous "what-if" kinds of operations through post-optimal analysis and does not 


require the planner to assign explicit weights to objectives. 


Three basic assumptions (Trueman, 1974, p. 231) in a linear programming model 


are: 


- proportionality (i.e., the amount of each resource used or requirement 
supplied) and the associated contribution to profit or cost must be 
exactly proportional to the value of each decision variable; 


. additivity (i.e., the total amount of each resource utilized or requirement 
supplied) and the total profit or cost are equal to the sum of the 
respective amounts. The two foregoing assumptions mean that all 
constraints and objective functions are characterized by linear 
relationships; 
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- certainty (i.e., the linear programming model is completely deterministic, 
having no stochastic elements) is assumed. 


Potential limitations of this technique are that objectives and criteria must be 
expressed in quantitative terms. The variables must be linearly related in terms of 


resource usage and objective contribution to the specified objectives. 


Goal programming: Goal programming is a decision aid for solving problems with 
multi-conflicting objectives through the use of a system of priorities (Lee 1972, 

p. 22). Low-order objectives are considered only when high-order objectives are 
satisfied or have reached a point beyond which no further improvements are desired. 
Instead of trying to maximize or minimize the objective function directly, deviations 
from goals, given a set of constraints, are minimized. Goal-programming problems 
are always minimization problems. Deviations from the highest priority goal are 
minimized to the fullest possible extent, followed by the minimization of deviations 


from the next goal, and so on. 


Goal programming shares the same assumptions as linear programming: 
proportionality, additivity and certainty. However, whereas linear programming 
optimizes one objective function, goal programming optimizes several conflicting 
objectives in the order of their importance. In linear programming, goals and sub- 
goals have to be quantified and reduced to one composite function, whereas in goal 
programming, this unidimensionality of the objective function is eliminated by 


treating goals and sub-goals separately. 


In the linear-programming approach, the values of choice variables are dictated by 
the objective-function criterion and tend to "drive" the value of the slack variables, 
whereas in goal programming, deviational variations "drive" the values of the choice 
variables. In linear programming, the relationships of the variables are expressed 
using cardinal numbers. In goal programming, sub-goals may be stated in terms of 


upper or lower limits. Further information is provided in Boritz (1983b). 


The primary advantage of goal programming is that it increases the dimensions of 
the objective function by accommodating multi-conflicting objectives; for example, 
Weeling (1977) applied the technique to the analysis and reconciliation of conflicting 


objectives of job productivity, human-resource development and individual satisfaction. 
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In summary, it is a flexible technique for decision problems that involve conflicting 
objectives, allows for an ordinal solution (Lee 1972, p. 22), and satisfies goals in 
their order of importance. It does not require that all goals be expressed in 
monetary terms or that they be reduced to a composite function. However, the 
variables must be linearly related in terms of resource usage and contribution to 

the specified objectives. The process of deriving the optimal solution is complicated 


and time-consuming and requires computer aids. 


Humans as Synthesizers 


Some problems are so complex that management science models simply cannot 
solve them within any reasonable time frame. They require human expertise 
because, despite limitations of cognitive processes, there are human experts who 
have found ways of successfully solving complex problems. There are essentially 


three approaches which may be used: 


% consult with a single human expert (e.g., a consultant), 
® consult a number of human experts and then aggregate their opinions, 
& rely on expert systems. 


Solomon (1982) studied probability assessments by individual auditors and audit 
teams (i.e., involving some degree of face-to-face interaction) as well as comparing 
human teams against "mathematically" formed teams (i.e., combining responses 
from non-interacting individuals into summaries as if they were done by groups). 

He found that the prior probabilities assessed by audit teams were more extreme 
and exhibited greater consensus;~? however, they were also less accurate than 
individual judgments. The comparison of interacting teams against statistically 
combined judgments to represent non-interacting group judgments indicated that 


interacting teams outperformed the statistically formed "groups". 


25 This is consistent with findings by Reckers and Schultz (1982). In addition, 
Reckers and Schultz found that interacting groups had greater confidence in 
their judgments (although this may be 'false' confidence). 
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The problems to be solved are many, but human experts are few. It is not always 
possible to find a human expert to act as a consultant in a difficult problem area. 
This accounts for the current emphasis on research into expert systems, since a 
single expert system can be used by many interested parties. Expert systems are 
discussed in Section Three of this chapter. While their applications to auditing are 


only now being explored, they hold great promise. 
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SECTION THREE: DECISION SUPPORT AND EXPERT SYSTEMS 


Complex networks of semi-structured tasks represent an ideal setting for the 
implementation of decision support systems (DSS). Johnson et al. (1979) suggest 

that through the use of decision support systems it is possible to transfer certain 
capabilities from those who have them to those who do not. Within the context of 
such support systems, specific techniques for enhancing information search and 
problem-structuring would play prominent roles. The goal of DSS and expert systems 
would be to help less expert professionals (e.g., students, trainees, etc.) to improve 
the quality of their judgments and decisions, and eventually, to improve their 

general problem-solving skills as well. In addition, these systems could serve to 
enhance communications; e.g., between the person who does the initial work and 


the various subsequent reviewers. 


In keeping with the foregoing discussion of assisted professional problem-solving, a 


decision support system could: 


@ help structure and focus the professional's cognitive image of the 
problem by incorporating strategies for (a) generating and pruning lists 
of hypotheses, and (b) grouping and organizing information; 


@ permit novices to mimic experts (within a certain range) by providing 
guidelines on problem-solving strategies; 


® provide the planning knowledge necessary for guiding the diagnostic 
process from beginning to end to help avoid or minimize premature 
conclusions, biased interpretations and judgments, and incorrect 
inferences; 


@ provide an organized database to extend specific case knowledge to 
permit improved lines of reasoning to be used; and, 


° simplify evaluation processes to help avoid faulty combinational 
systems. 


Automated Decision Support Systems 


There is no single widely accepted definition for decision support systems. Scott 
Morton (1971) defined "management decision systems" as, "interactive computer- 
based systems, which help decision-makers utilize data and models to solve 


unstructured problems". Keen and Scott Morton (1978) pointed out: 


\ 
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...semi-structured tasks, is where DSS can be the most effective. These 
are decisions where managerial judgment alone will not be adequate, 
perhaps because of the size of the problem or the computational 
complexity and precision needed to solve it. On the other hand, the 
model or data alone are also inadequate because the solution involves 
some judgment and subjective analysis. Under these conditions the 
manager plus the system can provide a more effective solution than 
either alone. (p. 86) 


Successful DSS, by bringing structure to at least part of managerial 
decision-making processes, shift the line dividing structured from 
unstructured problems so as to encompass more in the former domain. 
To this end some systems primarily store and retrieve data, and they 
leave to the user the important procedural aspects of decision-making. 
Others embody decision-making algorithms but leave to the user the 
task of collecting and distilling the data that must be provided for these 
algorithms. (p. 205) 


Sprague (1980, p. 5) stated the broad "charter" for decision support systems: 


Dedicated to improving the performance of knowledge workers in 
organizations through the application of information technology. 


However, Alter (1977, p. 49), referred to decision support systems as a "buzzword 


whose time has arrived". 


Alavi (1982) conducted in-depth interviews with senior-level executives who had no 
previous exposure to decision support systems. They were primarily concerned 
with handling conflicting objectives or criteria and making decisions on the basis of 
insufficient information. It was to these needs that they felt DSS could best 


contribute. 


Bonczek et al. (1981) described a computer-based decision support system developed 
for Xerox corporate planning (refer also to Seaberg and Seaberg 1973). This system 


was based on the following premise: 


...that it is possible formally and theoretically to describe the 
management process, that the resultant models can be programmed, 
and that a combined "man-model-machine" system can be used to make 
better decisions than could be made without the support of management 
science and computer science. (p. 52) 
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The advantages of such an approach, reportedly achieved by Xerox, were 


summarized as follows: 
e drastic reduction of analyst time to 1/8 of that previously required to 
manipulate data, 
° increased accuracy, 


e better specified forecasting logic to eliminate non-standard (and 
presumably less defensible) procedures, 


® drastic reduction in time required to generate plans, 
% internal consistency of data despite last-minute adjustments, and 
9 increased availability of time for concentrating on analysis. 


As discussed previously, the major goal of decision support is to transfer 
capabilities from those who have them (either individually or as groups) to those 
who do not. Within the context of such support systems, emphasis is placed upon 
specific techniques for enhancing information search and problem structuring 


behaviour. Keen and Scott Morton (1978, p. 2) described DSS as follows: 


e the impact is on decisions in which there is sufficient structure for 
computer and analytic aids to be of value but where managers' judgment 
is essential, 


e the payoff is in extending the range and capability of managers' decision 
process to help them improve their effectiveness, 


e the relevance for managers is the creation of a supportive tool under 
their own control, which does not attempt to automate the decision 
process, predefine objectives or impose solutions. 


The primary goal of such systems is to improve the quality of judgments and decisions. 


They may not, however, be equally effective at improving the DSS-independent 


problem-solving skills of their users. This may be an extremely grave consequence 
associated with DSS/expert systems and should be considered. 


Sprague (1980) specified a set of six capabilities or performance requirements for 
such systems. The first three pertain to the decision-making tasks, while the latter 


three pertain to the support mechanisms. Thus, a DSS should provide support for: 


Internal Audit Handbook 
Volume II, Part 2 


Chapter 6, Section 3 - 503 - 
e decision-making, but with emphasis on semi-structured and unstructured 
decisions, 
rY users at all levels, assisting in integration between the levels wherever 
appropriate, 
® decisions which are interdependent as well as those that are independent, 
e all phases of the decision-making process; i.e., goal formulation, 


information gathering, analysis, and action/choice, 


® a variety of decision-making processes, but not be dependent on any 
one; i.e., it should be "user driven", and 


@ ease of use. 


Alter (1980) identified the following six generic operations for decision support 


systems: 


retrieve a single item of information, 
perform ad hoc data analysis, 

produce standard reports, 

estimate consequences of proposed decisions, 


propose decisions, and 


make decisions. 


He ultimately grouped these into data-oriented and model-oriented functions (refer 


to Figure 15). 


Note that the foregoing objectives and principles are quite general and do not 
include specific technical implementation considerations. However, Johnson et al. 
(1979) emphasize that the foremost criterion for evaluating any decision support 
system is its acceptability and actual use, and these depend upon the developer's 


success in actually achieving the following objectives: 
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PHASES OF DECISION-MAKING AND DECISION SUPPORT BUILDING BLOCKS 
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Figure 15 
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A. Content 
1. relevance - right information provided, 
Zs availability - right information provided at the right time, 
3. specificity - information presented at the right level of detail, 
and 
4, justification - information presented for the right reasons. 


B. Processing 
ie accessibility - flexible access to and retrieval of information, and 
Pe efficiency - small user workspace required in interaction between 
user and system. 


C. Delivery 
i. comprehensibility - form known and acceptable to user, 
Le saliency - focus on important information, and 
a; useability - satisfy broad range of human-factors requirements. 


Mock and Vertinsky (1985) emphasize several additional concerns that DSS builders 


must take into account: 


) nature of the decision process, 

® resources and constraints which govern the behaviour of the intended 
users, 

® motivations of the intended users to use or misuse the system, and 

© organizational and environmental constraints (e.g., privacy, security, 


other statutory requests) to be satisfied. 


Examples of Decision Support Systems 


Although there have been great strides made to automate various aspects of 
auditing,-° few audit decision support systems currently exist. Most of the 
systems which do exist are either preliminary versions or are still under 


development; for example: 


26 or example, wide use is made of computer-assisted and statistical auditing 
techniques, automated audit working paper preparation systems, and spread- 
sheet packages. 
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a TICOM is a system for modelling an internal control system and then 
questioning the model for purposes of evaluating internal controls (Bailey et 
al. 1985). Using this system the control evaluation can be more rigorous and 
exhaustive, and the documentation can be more thorough than with conventional 


techniques. 


e CONTROL PLAN was developed by the accounting firm of Deloitte, Haskins 
and Sells (1985) to help an auditor record and evaluate features of client 
internal control systems. This microcomputer program compares pre-defined 
conditions that may exist within accounting and control systems against actual 
descriptive data entered by an auditor. This comparison can help the auditor 
determine whether conditions exist (alone or in combination with other 


conditions) to indicate specific potential control weaknesses. 


a CAPS is a computerized audit planning and risk analysis system (Boritz 1984). 
It permits directors of internal audit departments to identify risk factors 
which would be useful to prioritize audit units, evaluate their relative 
importance, set priorities within audit units and create long-term audit 


coverage plans. 


* ABC AUDIT PLANNER is an audit testing support system (Boritz 1985b). It 
permits an auditor to enter financial statement data, estimate or compute 
materiality and precision, evaluate overall audit risk, plan audit procedures 
by financial statement item by assertion, and compute sample size requirements 


if statistical samples are planned. 


Examples of decision support systems in other fields may be found in Decision 


Support Systems by Keen and Scott Morton (1978). 


Anatomy of a DSS 


Sprague (1980, p. 15) shows the general functional components of a decision support 


system, (see Figure 16) including: 
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A CONCEPTUAL MODEL OF A DECISION SUPPORT SYSTEM 


Legend: DBMS: Data Base Management System 
MBMS: Model Base Management System 
DGMS: Dialogue Generation Management System 


Figure 16 
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Databases: the data files that are stored in the computer. These files could be any 


combination of the following: 
- "live" transactions files generated by the organization's day-to-day 
activities, 


- databases created for the application of the specific decision support 
systems, 


- databases purchased from outside the organization. 


Database management software (DBMS): the programs that "manage" the databases, 


including: 
~ combining a variety of data sources through a data capture and 
extraction process, 
- adding and deleting data quickly and easily, 
~ portraying logical data structures in user terms, and 


- handling the user's data inquiry requests. 


Model bases: the various models (programs) that can retrieve, aggregate, and/or 
analyze the existing databases and/or additional inputed data. Some of these 
models allow the user to perform "what-if" analyses; i.e., the user can change the 


values of some variables to determine their impact or the model's sensitivity. 


Model base management software (MBMS): programs, similar to the DBMS, that 


"manage" the model bases similar to the DBMS, including: 
- cataloging and maintaining a wide range of models, supporting all levels 
of management, 
- creating new models quickly and easily, and 


~ interrelating these models with appropriate linkage through the 
databases. 


Dialogue generation and management software (DGMS): the programs that provide 


the interface between the users and the system, including: 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 6, Section 3 - 509 - 
- what the user "sees" (based on the display or presentation language), 


- what the user can do (based on the action language), and 


- what the user must know (the required user knowledge base). 
The DGMS should: 


- handle a variety of dialogue styles, 


- shift among the dialogue styles according to the user's choice, 


accommodate user actions in a variety of media, 
- present data in a variety of formats and media, and 


~ provide flexible support for the user's knowledge base. 


Relationship of Decision Support Systems, Computers, Management Information 
Systems and Management Science/Operations Research 


Although a decision support system could exist as a manual system, most researchers 
view decision support systems as computerized systems. Alter (1977, p. 40) classified 
business computer applications into electronic data processing (EDP) systems and 


decision support systems, where: 


® EDP systems are designed to automate or expedite transaction processing, 
record keeping, and business reporting. EDP systems emphasize clerical 
activities and are designed for processing efficiency. Data input and 
resulting output reports are performed on a relatively structured 
schedule. 


° Decision support systems are designed to aid decision-making and decision 
implementation. Decision support systems are for management and 
planning activities and are designed more toward overall effectiveness. 
The output of decision support systems is more on an on-demand basis. 


In a given organization it may be difficult to distinguish the EDP system and 
decision support systems. Both may share the same computer and some of the 


same files (databases) and report generators. 
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According to Keen (quoted by Bedard et al. 1983, p. 7): 


Some researchers view decision support systems as a subfield of MIS, 
while others regard it as an extension of Management Science techniques. 
The former see Decision Support as providing managers with access to 
data and the latter as giving them access to analytic models. 


Although the two concepts seem to overlap to some degree and Management 


Information Systems (MIS) predates DSS, Sprague (1980, p. 7) states that: 


Decision support systems are not merely an evolutionary advancement 
of EDP and MIS, and they will certainly not replace either... Nor is 
it... aimed exclusively at top management... It is, rather, another 
powerful weapon in the information technology arsenal, aimed at 
improving the effectiveness of managers... 


During the "golden years" of MIS (Naylor 1982) which continued to the early 1970s, 
some people envisioned a computer terminal available to every top executive. At 
minimum the executive could instantly make enquiries about the status of various 
aspects of the company such as sales trends for a particular product or the 
productivity of a particular manufacturing plant. Some foresaw even more 
sophisticated uses for DSS. For example, when faced with a problem, an executive 
would input the problem (in English) into the computer via the terminal and the 
computer would, in turn, present a suggested solution to the executive who would 


evaluate it and then take action. 


As previously described, the DSS framework would incorporate both MIS and 
Management Science/Operations Research (MS/OR). MIS is responsible for the 
information gathering and management phase, while MS/OR provide the models for 
enhancing the action choice phase. In addition, Sprague touches on the potential 
for incorporating elements drawn from artificial intelligence (AI) research into 


DSS; i.e., expert systems. 


Some pitfalls may be present for expert systems and DSS. A great deal of 
disenchantment followed MIS's enthusiastic introduction when MIS failed to perform 


as expected. Sprague (1980) addressed this concern when he stated: 
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The only hope for avoiding this wide swing in expectations is a realistic 
appraisal of what the decision support systems concept is, and what it 
can do. 


Naylor (1982) criticizes the entire DSS field asserting that: 


It exists primarily in the minds of academic visionaries and overly 
aggressive sales and marketing people. (p. 94) 


Sprague (1980) warns of "technology pushers" who concentrate on seeking problems 


which are susceptible to the tools they know how to use. 


Relationship of Decision Support Systems and Expert Systems 


Bonczek et al. (1981) conclude that decision support systems can be classified in a 
two-dimensional space (refer to Figure 17), where one dimension (e.g., the x-axis) 
represents the method used for directing data retrieval and the other dimension 
(e.g., the y-axis) represents the method used for directing computational procedures. 
The origin of the axes represents systems where the user states the retrieval 
procedure or computational procedures explicitly. The extreme positions of both 
axes involve a statement of the problem by the user rather than a definition of the 


problem-solving procedure. 


Needless to say, most (current) decision support systems fall in the mid-ranges of 
the graph. Here the user must specify what information is to be produced or which 
models are to be applied, but does not need to specify the detailed procedures to be 
used for retrieving the information or applying the models. As one moves to the 
extreme (northeast) positions represented by the axes in Figure 17, one enters the 


realm of expert systems. 


Expert Systems 


Expert systems are computer programs which emulate the problem-solving knowledge 


and skill of human experts. According to Feigenbaum (quoted by Gevarter, 1982, 
p. 2): 
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CLASSIFICATION OF DSS 


Implicit Detail 


Computational DSS 
Procedures 
Programs 
and DBMS 
Files 
Explicit Detail Data Retrieval Implicit Detail 


Prodecures 


Legend: GPS: General Problem Solver 
DSS: Decision Support System 
DBMS: Data Base Management System 
ES: Expert System 


Figure 17 
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An "expert system" is an intelligent computer program that uses 
knowledge and inference procedures to solve problems that are difficult 
enough to require significant human expertise for their solution. The 
knowledge necessary to perform at such a level, plus the inference 
procedures used, can be thought of as a model of the expertise of the 
best Practitioners of the field. 


The knowledge of an expert system consists of facts and heuristics. 
The "facts" constitute a body of information that is widely shared, 
publicly available, and generally agreed upon by experts ina field. The 
"heuristics" are mostly private, little-discussed rules of good judgment 
(rules of plausible reasoning, rules of good guessing) that characterize 
expert-level decision making in the field. The performance level of an 
expert system is primarily a function of the size and quality of the 
knowledge base that it possesses. 


In the past, expert systems were only known to computer scientists engaged in 

artificial intelligence (AI) research; but recently, they have been popularized (e.g., 
refer to Feigenbaum and McCorduck 1984, Longair 1983, Alexander 1982, Webster 
and Miner 1982). In fact, the terms "expert system" and "decision support system" 


are used quite casually these days, particularly in software product advertisements. 


In the 70's, it became apparent that database search strategies alone, 
even augmented by heuristic evaluation functions, were often inadequate 
to solve real world problems. The complexity of these problems was 
usually such that either (1) a combinatorial explosion occurred that 
defied reasonable search times, or (2) the ability to generate a suitable 
search space did not exist. In fact, it became apparent that for many 
problems, expert domain knowledge was even more important than the 
search strategy (or inference procedure). This realization led to the 
field of "Knowledge Engineering", which focuses on ways to bring expert 
knowledge to bear on problem solving. The resultant expert systems 
technology, limited to academic laboratories in the 70's, is now becoming 
cost-effective and is beginning to enter into commercial applications. 
(Gevarter, 1982 p. 1) 


In The Dragons of Eden Sagan (1977, p. 236) has commented that, "The next major 

structural development in human intelligence is likely to be a partnership between 

intelligent humans and intelligent machines." More specifically, he asserts that "in 
reasonably restricted contexts the human use of artificial intelligence seems to be 

one of the two practicable major advances in human intelligence available in the 


near future" (p. 222). 
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A fundamental issue in expert system development is whether the problem requires 
expertise. As noted earlier, decision-making may involve problems that are highly 
structured or highly unstructured. Highly structured problems are often subject to 
algorithmic solutions such as those proposed by management scientists. Less well- 
structured problems typically do not have algorithmic solutions. Instead, the 
solutions often depend on heuristics developed through years of study and experience 


-- in short, expertise. 


Hayes-Roth et al (1985) say that, "...the problem should be nontrivial but tractable, 
with promising avenues for incremental expansion". Hayes-Roth et al. (1983, p. 14) 
summarize 10 generic tasks requiring expertise, hence providing opportunities for 


expert system applications (refer to Table 3). 


Expertise enters into the execution of thése tasks as a result of the expert being 
exposed to numerous instances of a particular problem and learning how to perform 
these tasks well. For example, a physician's expertise arises from, "the large 
collection of empirical associations he or she accumulates by virtue of experience 
in the field" (Davis 1982, p. 4); i.e., an expert physician learns rules or heuristics 


over time about a particular disease(s) by being exposed to numerous cases. 


The range of behaviours exhibited by human experts carries implications for some 
of the engineering principles used in expert systems, as well as providing a standard 
against which the behaviour of expert systems could be evaluated. According to 


Davis (1982) experts do the following: 


discover that a problem exists, 

structure ill-structured problems, 

solve problems, 

explain the result or the method used to obtain it, 
learn from the case, 

restructure their knowledge, 


change or break rules when necessary, and 


get help when needed. 
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Task 


Interpretation 
Prediction 
Diagnosis 
Design 
Planning 
Monitoring 
Debugging 
Repair 
Instruction 


Control 
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Table 3 


Tasks requiring Expertise 


Problem Addressed 


Inferring situation descriptions from sensor data 
Inferring likely consequences of given situations 
Inferring system malfunctions from observables 
Configuring objects under constraints 

Designing actions 

Comparing observations to plan vulnerabilities 
Prescribing remedies for malfunctions 

Executing a plan to administer a prescribed remedy 
Diagnosing, debugging and repairing student behaviour 


Interpreting, predicting, repairing and monitoring system 
behaviours 


Source: Hayes-Roth et al. (1984, p. 14) 
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No expert system in existence today can fulfil such stringent requirements; 
however, as demonstrated below, existing expert systems have achieved significant 


levels of performance. 


Examples of Expert Systems 


© MYCIN was developed by Shortliffe (1976) for diagnosis of infectious diseases, 
particularly blood infections and meningitis infections and for prescribing 
treatment. MYCIN uses a four-stage decision process. First the system 
determines if the patient's condition, caused by bacteria, is critical. Secondly, 
based on the clinical information and the preliminary results of the culture, it 
identifies the organism. In the third stage, it considers potentially useful 


drugs, and finally, it recommends the best drug for this particular patient. 


® PROSPECTOR is another well-known system that was developed to aid 
geologists in the exploration for different kinds of ore deposits. Like MYCIN, 
the system gives the rationale for its conclusions and certainty (degree of 
confidence) factor. Some comparisons between the system and the human 
expert have resulted in an accuracy within seven per cent of the human expert. 
In addition, the system has also identified the location of a previously unknown 


high-grade ore deposit. 


© XCON (formerly, RI) configures customer requests for VAX computer systems 
at Digital Equipment Corporation. XCONS's input is a customer's order, and 
its output is a set of diagrams displaying the spatial relationships among the 
components on the order. These diagrams are used by the technicians who 


physically assemble the system (Harmon and King 1985, p. 155). 


C) DENDRAL is a chemistry expert system which supports hundreds of 
international users daily in chemical structure elucidation from mass spectra 
data (Hayes-Roth et al. 1983, p. 6). 


8 INTERNIST/CADUCEUS is the most knowledge-intensive expert system in 
existence. It embodies more knowledge of internal medicine than any human 


and can correctly diagnose complex test cases that stymie human experts. 
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"It covers more than 80% of all internal medicine; its knowledge base 
encompasses about 500 diseases and more than 3,500 manifestations of 
disease." (Feigenbaum and McCorduck 1984, p. 68.) 


@ PUFF integrates knowledge of pulmonary function disease with a previously 
developed domain-independent expert system for diagnostic consultations and 
now provides expert analyses at a California medical centre (Hayes-Roth et 
al. 1983, p. 6). 


In accounting and auditing, some systems have also been developed, or are being 
developed, that use a MYCIN or PROSPECTOR inference system. They include 
TAXADVISOR (Michaelsen 1984), AUDITOR (Dungan and Chandler 1985), and EDP 
AUDITOR (Hansen and Messier 1983). 


" TAXADVISOR is an individual income and transfer tax planning system that 
makes recommendations on the client's estate investment portfolio to 
maximize approximately the wealth transferred by the client at death. The 
system was evaluated by two experts from public accounting firms and was 


found to perform as well as tax experts. 


) AUDITOR is a program that assesses the adequacy of a client's allowance for 
bad debts. The system was evaluated by comparing its decision with an expert 


for 11 real world cases. Agreement between the two was 91 per cent. 


e EDP AUDITOR is a knowledge-based expert system that assists the auditor in 
the evaluation of the computer controls in a computer environment. The 
system is based on ALX which is expert system software developed from the 
PROSPECTOR system. EDP AUDITOR is still in its early developmental 


stage and consequently has not been validated. 


Anatomy of Expert Systems 


Bonczek et al. (1981) identify three principal components of decision support/expert 


systems: 
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- a language system, 
- a knowledge system, and 


- a problem-processing system. 


A language system may encompass both data retrieval languages and computational 
languages but is not concerned with the interfacing of models and data. It isa 
means of communication between that user and the DSS. A knowledge system 
contains knowledge about the user's problem domain, which may include both facts 
and rules for using facts, represented in such a way as to be accessible by the 
problem-processing system. The problem-processing system may be viewed as the 
interfacing mechanism between the knowledge system and the user's problem 
expressed via the language system, (also called inference procedure/engine). The 


key elements of an expert DSS are diagrammed in Figure 18: 


@ the external environment of the system, including in particular, the 
knowledge domain of interest, 


8 the knowledge base of the system, consisting of two main components; 
i.e., (i) domain-specific facts and (ii) heuristics, the set of procedures 
for solving problems in the domain by using the facts, 


@ the expert problem-processing program which guides the use of facts 
and problem-solving (also called the inference engine or rule 
interpreter), 

r) the language system interface whereby the human user interacts with 


(the system developer via) the expert program, 


r) the human user who utilizes the expert system for enhancing judgment 
and amplifying problem-solving skills, and who perhaps provides new 
information to the system. 


In the background are the equipment, software (i.e., language) and media for 
developing and operating the expert system. In the background as well is the system 
developer (the knowledge engineer) who gathers and analyzes information, codes 


and physically controls the functions of the expert system. 
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COMPONENTS OF EXPERT SYSTEMS 


HEURISTICS 


PROBLEM PROCESSING SYSTEM 


SYSTEM/USER INTERFACE 


USER 


Figure 18 
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According to Gevarter (1982), there are three different user-modes for an expert 
system in contrast to the single mode (getting answers to problems) characteristic 


of the more familiar type of computing: 


% getting answers to problems -- user as client, 
* improving or increasing the system's knowledge -- user as tutor, and 
a harvesting the knowledge base for human use -- user as pupil. 


An expert system acts over time as a systematizing repository of the knowledge 
accumulated by many specialists of diverse experience. Hence, it can ultimately 


attain a level of consultant expertise exceeding that of any single one of its "tutors". 
An expert system differs from conventional computer programs (Duda 1981, p. 242). 


... There is a clear separation of general knowledge about the problem (the 
rules forming a knowledge base) from information about the current problem 
(the input data) and methods for applying the general knowledge to the problem 
(the rule interpreter). 


In a conventional computer program, knowledge pertinent to the problem and methods 
for utilizing this knowledge are all intermixed, so that it is difficult to change the 
program. In an expert system the program itself is only an interpreter (or general 
reasoning mechanism) and, ideally, the system can be changed by simply adding or 


subtracting rules in the knowledge base. 
Concluding Remarks about Decision Support and Expert Systems 


Because the entire field of expert systems research and development is in a rapid 
process of evolution, speculating about the ultimate capabilities of such systems is 


not a very useful exercise. However, several points are noteworthy. 


e In general, the poorer the structure of a task environment, the more 
valuable the judgment of an expert, even if the expert is merely a series 
of computer-coded productions (i.e., (if...then) statements of fact, 
and/or problem-solving tactics or heuristics). Gorry and Krumland 
(1983) emphasize the role of DSS in structuring ill-structured problems. 
Expert systems are an extension of this structuring activity. 
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8 Expert systems generally focus upon relatively narrow problem domains 
rather than embodying expertise of a general or multi-purpose nature. 
For example, MYCIN is restricted to the diagnosis of meningitis. 
PROSPECTOR is restricted to the evaluation of ore-deposits. Of course, 
there is no reason other than feasibility that a number of such systems 
should not be connected; one "piping" its output to become another's 
input. 


e Expert systems depend critically upon the existence and identification 
of computer-codifiable expertise in the problem domain, as well as on 
the existence and application of appropriate expert system development 
tools and methods for eliciting, representing, coding and subsequently 
accessing the expertise. Neither of these pre-requisites should be taken 
for granted; i.e., identifiable expertise may not exist and/or the tools 
for representing may not be available. 


e Expert systems are not value-free. Their widespread adoption and 
proliferation carries both payoffs and penalties; but, the latter are 
hardly ever discussed. For example, once an expert system is created 
and put into use, where will the next generation of expert human come 
from? On the other hand, such systems can take over the human 
consultant's role in a decision task, making the expert system itself a 
reliable and extremely powerful part of a larger decision support 
network, aimed at expanding the problem-solving capabilities of an 
organization, a team, or an individual. 


The field of auditing is currently undergoing rapid technological change. The use 

of quantitative models is expanding. Associated with their growth is an increased 
emphasis on the use of computer-assisted tools and techniques for automating a 
variety of diverse audit tasks and amplifying the problem-solving capacity of audit 
professionals. Although by no means the only, or even necessarily, the most 
significant developments in the audit profession, the trends towards decision- 
support and expert systems are evidenced by the large sums of money being invested 
in these facilities by all the major public accounting firms, several governmental 


bodies and professional associations. 


It is difficult to imagine what the character of the auditing profession will be like 
when these tools become widely available and used. It appears certain, however, 
that we are at the forefront of a great wave of automated decision support systems 
aimed at enhancing virtually all aspects of audit activity. This suggests that many 
corresponding changes will be required on the human front. In some cases, jobs will 
be eliminated. In others, jobs will be enriched through the interaction of auditors 
and decision-support/expert systems. Hopefully, the quality of auditing will be 


improved as a result. We shall see. 
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Appendix A 
THE KNOWLEDGE ENGINEER AT WORK 
(Excerpted from The Fifth Generation, Feigenbaum and McCorduck, 1984, pp. 87-89.) 


First, of course, she must persuade a human expert to agree to devote the 
considerable time it will take to have his mind mined. Experts, by their nature, 
are busy people, constantly being called upon to do just one more thing. But 
for a variety of reasons, experts can be persuaded to participate, and so the 
project begins. Once Nii has secured the expert's cooperation, she immerses 
herself in his field, reading college textbooks, articles, and other background 
material, in part to understand what the field is about, and in part to pick up 


the jargon that pervades every field. Now she is ready for the first interview. 


At the beginning, she asks the expert to describe what he thinks he does, and 
she also asks him to think about how he solves problems. She urges him to 
choose a fairly difficult problem to examine. Nothing makes everybody lose 
interest faster than an easy problem, and furthermore, an easy problem reveals 
little that's significant about someone's expertise. Nii's guideline is that 
although the problem under consideration should be nontrivial, it shouldn't be 
too hard, either. She generally prefers problems that will take humans a few 
hours to solve, because if a problem takes days to be solved by a human, it's 
probably too difficult or ill-defined to be engineered into an expert program 


using current AI techniques. 


Having collected this initial information, she brings it back to the other 
members of the team, the programmers. Though the programmers do the 
actual coding, it's up to the knowledge engineer to choose which of several 
available problem-solving frameworks inference procedures -- best suits the 
new domain. The programmers must get a first version up and running ina 
few days. Strangely enough, those first few days (as opposed to weeks) are 
critical psychologically for hooking the expert into the project. Experts, 
even as the rest of us, like their gratification sooner rather than later and are 
much likelier to continue to commit that precious resource, time, to a project 


they can see is making progress. 
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Of course, there are likely to be major flaws in the first version of the expert 
system. Perhaps the expert hasn't really articulated very well what he does. 
Perhaps he's been misunderstood. Perhaps -- and this is often the case -- the 
method he has claimed to use is a textbook fiction and has very little to do 
with his practice in the real world. He groans as he sees the program on 


display before him: "Not that way," he says. 


"Then how?" Nii asks. "Where did we go wrong?" 


But if he couldn't articulate it at first, perhaps he's no better at it now. Thus 
Nii asks him to talk his way through the model problem, verbalizing each step 
as he goes. This time, she observes, is usually very different from the textbook 


version he first gave of his problem-solving techniques. 


She watches him carefully. Sometimes he says he relies on data that in fact 
his eyes never come to rest on, or maybe she sees he uses them at a different 
stage from the one he says he does. All this must be integrated into a new, 
amended version of the expert system and brought back to the expert to 


validate or correct before his interest strays elsewhere. 


Nii says that during the interviews she is not necessarily listening to the facts 
the expert gives her so much as how he manipulates his knowledge. As he 
talks, she is systematically evaluating mentally the various AI knowledge 
representation and inference methods -- object oriented techniques, blackboard 
techniques, production rules, to name a few --seeing how well one or any of 
them matches the expert's behaviour. She asks questions: "Would this make 
sense?" "Can you do it that way?" These are not only to extract more 
knowledge from the expert, but also to test the model of his work she's building 
up in her own mind. Moreover, she has to determine whether this expert is 
special in his interpretations and assumptions, or whether there's uniform 
agreement in his field. When she compares his knowledge to textbook 
knowledge, she usually discovers that the textbook is so general it's nearly 
useless. Typically, an expert confronted with a textbook assertion will say: 
"That's true, but if you see enough patients/rocks/chip designs/instrument 


readings, you see that it isn't true after all. At this point, knowledge threatens 
to become ten thousand special cases. 
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There is, in addition, the problem of keeping the expert focused during the 
interview -- even the minds of experts will wander. One of Nii's tricks is to 
concentrate on the specific problem she has asked the expert to provide, the 
mode! problem that will not only keep everybody's interest lively, but will 
also serve as a good test for her own mode! of how thinking in the field is 
shaped. All these processes are repeated day after day, the expert being 
continuously presented with an updated version of the computer program that 


is intended to mimic his behaviour. 


Despite these good intentions and careful preparations, sometimes everything 
goes off on the wrong track -- the expert chooses an inappropriate problem, 
or the knowledge engineer chooses the wrong tools with which to express the 
process. Nii has written, "One of the difficulties of writing knowledge-based 
programs is that at least two parties are constantly shifting their points of 
view: the domain expert and the knowledge engineer. As the knowledge in 
the program accumulates and the problem becomes clearer, the knowledge 
engineer may find better ways to represent and process the knowledge. The 
resulting behaviour of the program may inspire the expert to shift his view of 
the problem, creating for the knowledge engineer further problems to be 
solved. Development of expert programs involves a process of finding a 
workable relationship between experts and programmers and slowly evolving 


a program structure that will work." It's an elaborate and tricky pas de deux. 
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CHAPTER SEVEN 
AUDITOR-AUDITEE RELATIONS 
INTRODUCTION 


The internal auditor's role is complicated considerably by environmental, behavioural 
influences. Familiarity with these influences can help the auditor to prepare 
appropriately to carry out audit assignments in a way that will enhance the potential 


for successful results. 


In this chapter an attempt will be made to sensitize the auditor to the existence 
and implications of the various behavioural effects for the audit function, and to 
what actions the auditor might take to deal with them while maintaining audit 


effectiveness. 


Behavioural influences, referred to in this chapter as auditor-auditee relations, 

have several generic aspects and some that are process-specific. Accordingly, the 
discussion to follow is presented in four sections: two generic ones on role-related 
and organization-related influences (Sections One and Two); an audit process-related 
section (Section Three); and lastly, the implications of non-audit activities (Section 


Four). 


Although the chapter is generally entitled "Auditor-Auditee Relations", in the 
detailed discussions a distinction is made between the direct auditee and the ultimate 
client. This distinction is dealt with specifically in a subsequent section but is 


recognized throughout. 


The auditee's perception of the auditor has to be faced squarely at the outset. Given 
that the behavioural implications of the audit role are recognized by the auditor, 
much of the potential reaction (specifically negative reaction) of the auditee to the 
auditor can be neutralized or compensated for. Most of the relevant issues are 
raised; however, those that are covered in the literature are not dealt with 


exhaustively. 
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A Bibliography is provided for those who wish to delve further into this important 
aspect of auditing. A caveat regarding relevant literature is in order. Although 
relevant literature is quite extensive on some of the topics raised in this chapter, it 
is usually not presented in the audit context. Therefore considerable scope remains 


for auditors who wish to add to the audit body of knowledge in this area. 
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SECTION ONE: ROLE-RELATED INFLUENCES 


Relationship Between Staff and Line Roles 


Internal Audit as a Staff Function 


In ideal terms, the distinction between line and staff is very clear. Line managers 
make the decisions regarding production and delivery of the product or service of 
the organization under consideration, and staff managers advise the line officials/ 
managers in making those decisions. As might be expected, the real world does not 


behave in so clear-cut a manner. 


In practice, the distinction between line and staff is not one of "white" and "black", 
but rather one of "shades of grey". For example, is purchasing part of the production 
process? What about sales, marketing, research and development? Even those 
functions traditionally seen as staff functions (e.g. finance, personnel, EDP) have 
both line and staff roles; staff with respect to the managers served, and line with 


respect to their own function (responsibility centre). 


The internal audit function is no exception in this respect. However, in terms of 
its role vis-a-vis the auditee, it resembles the role of a staff specialist rather than 
a function specialist such as the comptroller. Whereas the function specialist (e.g. 
the comptroller) has responsibility for, and is accountable for, financial systems 
that are used by line managers, as well as for provision of financial advice, the 


internal auditor provides advice only. 


In any case, the key distinction is made on the basis of who makes the final decisions 
in the area under consideration. In the case of the auditor, it is the auditee who is 
expected to make the decision, regardless of who the auditee is (i.e. line or staff 
manager or, at the highest level, the client). The auditor is merely one of the 
information sources which impinges on the auditee manager's decision-making 


process. 
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There are three main messages to be received from the foregoing discussion: 


2 the role of the internal auditor is that of adviser; either the auditee 


manager or client makes the final decision as to disposition of findings 


and related recommendations, depending on the level of the finding; 


@ the auditor's normal output is one source of input to the auditee's 


decision-making process and, therefore, the auditor's activities and 
results should be attuned to the auditee's key decision areas if that role 


is to remain relevant; and 


2 audit heads or managers perform line-type activities with respect to 


audit operations. 


Historical Role of Auditors 


The earliest application of an audit-like role was in conjunction with tax 
collection and consisted primarily of fraud or defalcation deterrence and 
detection. To some degree, this role is retained by modern auditors (whether 
external or internal), however, it is no longer the primary one. Nevertheless 
because of this historical association a residual image of the auditor as "police 
officer" persists in the mind of the auditee and public at large. This fact 


needs to be kept in mind in planning for and implementing audit activities. 


In the private sector, external auditing is focused on financial statements and 
underlying controls. Private sector internal auditing, although broader in 
context, is concerned primarily with financial and managerial accounting by 
virtue of the fact that private sector organizational goals revolve around 

some measure of return on investment, which is usually represented in financial 


or dollar terms. 


The auditor's image of "police officer" is both archaic and current. Archaic in that 


the fraud-detection role is only a minor aspect of the modern internal auditor's role 


Set. 


However, to the degree that the auditor continues to review, evaluate and 


comment on the auditee's performance (whether directly or indirectly), a surveillance 


connotation is unavoidable, even necessary (e.g. for deterrence purposes). 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 7, section 1 - 547 - 


& Given the unavoidable element of surveillance in the auditor's role, the only recourse 
left to the auditor is to recognize the implications for auditor-auditee relations and 


its impact on the efficiency and effectiveness of the audit function. 


A police officer can establish a reputation for balance, fairness, humanity and 
competence, in which case a considerable measure of cooperation is likely (e.g. the 
British "Bobby"). On the other hand, given the public's natural mistrust of all 
authority, it is much easier for a police officer to be seen as an arbitrary, biased, 


"bully" who uses and enjoys the superior role that the law allows. 


Similarly, the auditor can establish the reputation of an ally or at least of a neutral 
observer. However, the more likely image will be that of police officer, in the 
most negative sense. This image will have to be recognized and every effort made 


to replace it with one less hostile, if the auditor is to be effective. 


To establish a strategy for neutralizing the typical negative image of the audit 


function, the elements of that negative image must be identified, along with their 


) sources, and individually addressed. Some of these are enumerated below (see 
Mints): 
Elements Source 
Fear - fear of the unknown (never faced an auditor before); 


- historic reputation of audit function; 

- auditee is aware of weaknesses that have not been 
made known to superiors and fears being found out; 

- fear of fraud or defalcation being detected; and 


- fear of disturbance of the status-quo. 


1 Mints, Frederic Ernest, The Effects of the Internal Auditors' Behavioral 


Patterns on His Relationships with Operating Personnel. 


®) 
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Suspicion/ 
Distrust - auditors motives unknown (first time auditee); and 


- natural suspicion of any outsider. 


Resentment - of any control role; 
- of the interposition of a powerful third party into the 
normal supervisor- subordinate relationship; and 
- of the auditor getting credit for identifying problems/ 
putting forward ideas already well known to the auditee, 


without giving due credit. 


Given the natural tendency of the auditee to harbour a negative image of the auditor, 
and considering the degree to which the efficiency and effectiveness of the audit 
process depend on good communications, there is obviously a need, on the part of 

the auditor, for a very proactive effort aimed at neutralizing that negative image. 
Otherwise, the auditor can expect considerable distortion and reticence in the 
communication process, with consequences for the audit process in terms of both 
efficiency and effectiveness. Efficiency suffers in that suspected reticence and 
distortion in communication requires auditors to do more verification while effective- 
ness has to do with the potential for misleading or missing information leading the 


auditor to erroneous conclusions. 
The above list of elements, which contribute to the auditor's negative image, is 
stated from the auditee's point of view. There are a number of ways in which the 
auditor may contribute directly to this negative image by: 
- exhibiting a superior, arrogant, discourteous or condescending attitude; 
- being ruthless and inflexible; 
~ being manipulative; 


- being biased/unfair; 


- being inadequately prepared (i.e. demonstrating incompetence); 
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- not distinguishing between significant and petty findings; 
- presenting results in a negative, blame-oriented manner; and 


- not giving credit where credit is due and not explaining mitigating 


circumstances. 


Once the factors leading to a less-than-flattering image of the auditor are 
enumerated and digested, appropriate action can be initiated to counteract them. 
As may be seen by the very nature of the factors involved, the task is an unceasing 


one. 


Some of the results of a research study (Doctoral dissertation), carried out by 
Frederic Ernest Mints?, should be helpful in determining where efforts to improve 
the auditor's image should be placed. For example, it was found that reactions to 
the auditor depend on the level and attitude of the superior. The higher the level, 
the less negative the image (possibly because the higher-level manager does not 
feel directly under attack and, so, less threatened). Also, it was found that if the 
superior had a positive reaction to the auditor, the subordinates were also more 
positive. However, subordinates involved in technical activities tended to be more 
antagonistic than regular line employees, probably because it is more likely that 


their ideas are being attacked in audit findings. 


As to the second issue, that of private versus public sector auditing, the following 
factors need to be considered. Public sector audit has recently had its scope of 
activities broadened to include some version of comprehensive or broad-scope 
auditing (in the federal government: the OAG in 1977 and Interna! Audit in 1982). 
This extends the auditor's interest to assessment of economy, efficiency and 
effectiveness of all operations, at least in terms of the controls associated with 


these types of performance requirements. 


2 Ibid, pp. 33-37. 
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Broad-scope notwithstanding, the auditee population and the public in general, still 
see the auditor as essentially financially oriented and preoccupied with dollars and 
compliance. In this context compliance tends to be construed as rules or means, 


as opposed to results or ends centred. 


This image persists, in some cases, with good reason. Many audit groups, although 
nominally auditing according to broader mandates, are populated with former 
financial auditors whose expertise is in the financial area of their host organization. 
Furthermore, even when auditing in non-financial areas, there is a tendency to 


concentrate on financial aspects of the operations under audit. 


The financial/compliance image can be mitigated with a combination of initiatives. 
The first initiative is communication, which serves to sensitize managers to the 
true nature, extent and thrust of modern internal auditing, and to elicit their 
concerns and support. The second is the demonstration before, during and after an 
audit assignment, of an attitude of empathy and constructiveness and an aura of 
knowledge, competence and objectivity. Finally, the results produced must bear 


out the constructive attitude and quality of output promised. 


In summary, internal auditors must be conscious of the inherent negative image that 
they suffer due to the historical role of auditors and, therefore, be constantly 


working to mitigate it. 
Independence and Objectivity 


Independence and objectivity are dealt with extensively, and relatively consistently, 
in the audit literature, and it is not the intention to repeat that content here. 
However, there are some aspects of the subject(s) which bear clarification and 
elaboration. For example, the concepts of independence and objectivity are dealt 
with as if they were interchangeable. Also, the two concepts are typically dealt 
with as if they had intrinsic value which is independent of the auditor's role. This 
is, of course, true for some, but not all, of the factors involved, as will be demon- 


strated in the following discussion. 
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In dealing with the concepts of independence and objectivity, or for that matter all 
the concepts in this chapter, it is important to bear in mind the ultimate purpose of 
an audit activity. As displayed in Figure |, the effectiveness of internal audit 

activity is ultimately determined by the degree of performance improvement in the 


auditee's area of responsibility which may be attributed to audit activity. 


The degree of performance improvement that is attributable to audit activity is, of 
course, difficult to determine since auditee performance is dependent upon a number 
of variables, only one of which is internal audit. Furthermore, internal audit may 
not be the most influential one. Nevertheless, the perspective that this approach 
gives to the auditor is important; i.e. the ultimate purpose of audit activity is not 
superb reports but improved auditee performance. However, the importance of the 


report's quality should not be underestimated as a motivator. 


To achieve improved performance, the auditee must be motivated to appropriate 
action. This will only happen if the auditee's attention, and subsequent acceptance, 
is obtained. Attention is obtained by provision of information which is relevant, 
while acceptance depends on its credibility. Finally, credibility is affected by a 
number of environmental and individual (audit group, auditor) factors which affect 
the existence and appearance of independence and objectivity of auditors (see 
Table 1). 


In the discussion of independence and objectivity it is important to recognize the 
precedence relationship between them. The goal is objectivity (in fact and in 
appearance). Once this is clear, it is easier to think in terms of the actions one 


must take to achieve that desired end. 


Table 1 enumerates a number of the key factors affecting the auditor's independence 
or objectivity. It is divided into three major lists to focus attention on both their 
relationships and their differences in terms of the decisions and actions the auditor 


might take to enhance the credibility of the audit group and its products. 
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FACTORS AFFECTING AUDITOR'S CREDIBILITY 


(and ultimate effectiveness) 


Environmental Personal/Group 


Factors Factors 


Independence and 


Objectivity 


Relevance and Credibility 
Attention/Acceptance 


Action 


I 


Improved Auditee 


Performance 


Figure | 
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Factors Influencing Independence and Objectivity 


ENVIRONMENTAL FACTORS 


Policy/Standards 


(central agency) 


Audit Mandate 


(departmental) 


Reporting Relationship 


(organizational level) 


Function Organization 


(centralized/decentralized) 


Audit Committee 


Senior Management Support 


Physical Location 


Activities Assigned 


Supervision 


Reputation 


(of profession/group) 


INDIVIDUAL ATTRIBUTES 


Background 


Affiliation 


Intellectual Independence 


Personal Integrity 


Education/Training 


INDIVIDUAL ACTION 


Conduct 


Demonstrated 


Balance/Fairness 


Demonstrated 


Competence 


Personal Image 


Personal Reputation 
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The environmental factors list consists of preconditions for independence, in fact 
and appearance. Since audit literature deals adequately with these factors, they 
will not be discussed here except to reiterate that they contribute to, but do not 
guarantee, objectivity. Conversely, if one or more of those preconditions is not 
met, it does not preclude objectivity in fact, although it makes being seen to be 
objective more difficult. All of these factors should be seriously considered in 


establishing an internal audit function if its effectiveness is to be maximized. 


The second list, that of individual attributes, has implications primarily for internal 
audit human resource management. To the degree that individual attributes are a 
product of genetic and early environmental history, the only tangible, relevant 
action affected is recruitment. Although the required attributes are often difficult 
to detect through interviews and written tests, they should get due consideration. 
References and examples of past output are particularly relevant. Of course, the 
portion of the background that is the product of education and experience is subject 
to influence by judicious training and/or assignment initiatives. To the degree that 
competence, or the appearance thereof, is a contributor to objectivity, these initia- 
tives could be beneficial. In any case, they would certainly contribute to improved 


relevance. 


Affiliation has two aspects. The first, that of personal philosophy, school of thought, 
etc., is not subject to much influence, in the short term, and may be difficult to 

detect during recruitment. The second, that of professional, social or other affiliation, 
is more amenable to influence and should be considered by both auditor and audit 


manager for its potential impact on objectivity. 


The final list, individual action, is ultimately the most important since it reflects 
the factors with the greatest influence on objectivity in a specific instance (e.g. 
audit assignment). Auditors, being on "foreign territory" for the majority of their 
activities, must be particularly careful of their conduct. This includes attitudes 
displayed (e.g. arrogance, condescension, insensitivity), independence of thought 
and opinion, social relationship (e.g. favouratism, fraternization), and rigour (e.g. 


substantiation, thoroughness, logic). 
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Finally objectivity must be demonstrated in the auditor's equivalent of the "bottom 
line", the report (whether oral or written). This consists of two factors, content 


and presentation with obvious implication for the required skill set. 


Throughout all these considerations the adage about good reputations being difficult 
to build but easy to tear down is particularly relevant for auditors or, for that 


matter, to any person or group that depends on persuasion for effectiveness. 


The Auditor as Catalyst 


All the role-related concerns discussed thus far have implications for the auditor's 
role as ‘catalyst. Since the auditor does not make the change decision, it is the 
auditor's influence role that predominates. Influence is, of course, highly dependent 


on context, credibility and communication ability. 


There are, however, other factors to be considered. If, as advocated in the previous 
sub-section, the auditor's effectiveness depends on implemented changes to the 
auditee's processes which result in improved performance, then being a successful 


catalyst depends heavily on timing as well as content and persuasiveness. 


Timing, in this context, has its own set of implications. It depends on such factors 
as the organization's current conventional wisdoms, coalitions, opinion leaders, 
competing initiatives, capability to implement, capability to absorb, and compati- 
bility with thrusts and goals. Many of these factors are self-evident, once explicitly 
identified, but often neglected by auditors in the development and presentation of 


their results. 


There are three related aspects to take into account in ensuring that these factors 
get due attention. One is awareness of the factors, the second is willingness to 
take them into account and the third is capability to deal with them. Awareness is 
dealt with through appropriate training, methodology and supervision of the auditor. 
Willingness depends on incentive, and capability on training, experience and natural 


ability. 
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The key to the issue is willingness. This is primarily an attitudinal problem. There 
are many auditors who feel that their job is done when they have identified weak- 
nesses and written a report based on those findings. Implementation, not being 

their responsibility, is given little consideration. This is perhaps a carry-over from 
the external audit "attest" orientation. Whatever its source, it is a 
sterile/unproductive approach which promises little for the auditee besides criticism. 
What is more likely to impress the auditee is potential for marked improvement of 


material consequence in the performance of his/her operations. 


In summary, the auditor must go beyond sterile reporting of facts, no matter how 
relevant and material. Findings and associated recommendations must demonstrate 
the consideration of appropriateness, given the current conditions in the auditee 


environment; i.e. how likely is the recommendation to be implemented? 
Auditing versus Investigating 


The roles of auditor and investigator have many similarities, but also some important 
differences. The similarities and differences can be characterized in two general 


classes. Those having to do with results expected and those which concern process. 


Normal internal audits do not generally result in legal proceedings, and instances 
where special audits are initiated due to suspected fraud are even less frequent. 
However, since the possibility does exist, an awareness of the implications of audit 


evidence becoming the subject of legal proceedings is desirable. 


Although the results of an audit may be such that’legal proceedings can ensue 
directly from them, the more likely situation will require that further evidence, 
appropriate for use in a court of law, be gathered. This implies two points of 
difference, both having to do with the difference between audit and legal evidence. 
The evidence gathered depends on two factors which represent the two points of 
difference. One is mandate and associated powers (legitimacy), the other is 


process. 
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The auditor typically does not have a mandate to search personal effects, to talk to 
outside parties about activities of auditee personnel, take sworn statements, etc., 
nor is an auditor necessarily skilled in these areas. If the auditor were to engage in 
such activities, some difficulty could be expected in defending them in a court of 
law under cross-examination; for example, in a defamation of character suit. Also, 


an auditor does not usually have any legal or forensic training. 


There are cases where the two roles, that of audit and investigation,are combined 
into one composite group. For example the U.S. federal government has inspectors 
general who have such dual mandates. However, even in these cases a distinction 

is made between these two activities in terms of mandate, skills required and results 
expected. In these cases audit evidence may trigger investigative activity but 


investigation is treated as a separate process resulting in separate reports. 


Where the incidence of fraud, abuse and related activities is high, there may be 
advantages to having these two groups working closely. The main disadvantage 
would be that any tendency on the part of the auditee to view the audit function as 
a police role would be strengthened. This in turn would tend to detract from the 


constructive, collegial approach that modern internal auditors try to foster. 


In conclusion, although the two roles of auditor and investigator are similar, and in 
fact may share data, they are not identical. They typically differ in mandate and 
in process and require different skills. Also, combining the two roles would tend to 


reinforce the police image of auditing. 


The Impact of Other Audit and Quasi-Audit Groups on the Internal Audit Role 


Second to managerial monitoring, internal audit should be the main control oriented 
activity in a department or agency. This has implications for internal audit's role 
vis-a-vis other audit and quasi-audit activities, particularly those initiated by 
external agencies (e.g. OAG, central agencies). Also, the way in which non-internal 
audit groups carry out their role will inevitably affect internal audit itself, since 
auditees very often do not discriminate between internal auditors, external auditors 


and quasi-auditors. 
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There are then, three related factors to consider. The first is the co-ordination of 
activities such that a minimum of duplication of effort occurs. The second is 
internal audit's role of buffer for the host department and adviser to management 
regarding these other audit roles, activities and outputs, and third is that of 
differentiation of internal audit's role so that any negative experience on the part 


of the auditee is mitigated. 


In sum, internal audit has the responsibility to the limit of its mandate to minimize 
the disruptive effect that other audit and quasi-audit activities may have on depart- 


mental management. 
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SECTION TWO: ORGANIZATION-RELATED INFLUENCES 


Stability of the Auditee's Organization 


In any control activity, audit included, there is a presumption that the auditee's 
goals, processes and environmental conditions are stable; that is, they are either 
static or changing in a predictable way. In the latter case the predicted changes 
should be reflected in plans, processes and associated control objectives. If the 
processes are not stable (i.e. are in a state of unpredictable flux) then one cannot 


expect predicted performance criteria. 


From the point of view of auditor-auditee relations, it must be remembered that an 
unstable situation usually means that the whole auditee area is under considerable 
stress. The auditor, therefore, must first recognize this situation and proceed ina 


way that will not aggravate it. 


Although the foregoing assertions are self-evident, once stated, they are all too 
often ignored or given little weight in audit assignment plans and activities. To act 
upon these assertions requires a more detailed understanding of the auditee environ- 
ment; however, indications of such an understanding in the way in which the auditor 
plans, identifies weaknesses and makes recommendations is, typically, not evident. 
The predetermined control model should reflect any tendency to instability or 
variability in the auditee processes, but seldom does. 


3 mode! is 


In the case of instability, the concept of a predetermined control 
essentially irrelevant for these portions of the auditees process that are in a state 
of flux. If the system (process) or sub-system being controlled is behaving in an 

unpredictable way, it is unlikely that reliable control objectives can be established 


for the relevant controls. The best that an auditor could expect under the 


3 See Volume II, Part 2, Chapter 2, "Control: Concepts and Applications for 


Internal Auditors". 
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circumstances is some evidence that the auditee recognizes the existence of the 
instability and has, or is making, plans for bringing it under control. Given that 
stability can be predicted for some future date, the auditor would then arrange to 


reschedule the audit accordingly. 


Where the auditee process is stable but varying in some predictable way, the auditor 
would expect to find corresponding variations in control objectives for each relevant 
control such that contro! objectives parallel the state of the process being controlled. 
This is a rather sophisticated strategy, requiring equally sophisticated control and 


audit methods. 


The situation where its process is stable but varying in some predictable way can 
occur for two reasons. One is where the program or project being undertaken is 

such that the outcome is not known with certainty; i.e. a living experiment. This is 
not as uncommon as it sounds. Many socio-economic policies/strategies are not 
testable/verifiable in a controlled-experiment environment, in which case the 

only recourse is a live experiment. In this case both the result and the delivery 
process will be subject to controlled variability (i.e. variation by design), and the 
control framework would be expected to vary accordingly, in both form and substance 
(goals). Also, since the delivery process has not stabilized (is expected to vary), 

both it and its corresponding controls may be expected to be more flexible, less 


permanent and often more costly. 


The second possibility is where the expected result is reasonably well defined but 
the most effective method of delivery is uncertain. Here, as in the above case, the 


auditor would expect to find a more variable control framework. 


Generally, unless the assignment plan and the predetermined control model, along 
with its associated audit program, reflect instabilities in the auditee processes 
and/or environment, the audit process and results are unlikely to take account of 


them and are therefore very prone to be judged insensitive, if not irrelevant. 
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Management Style 


There are many aspects of management style that could be discussed from the 
point of view of management theory, however, for audit purposes, only those aspects 
most relevant will be discussed here, namely those having to do with management 


control. 


In most large organizations stability is central in the consideration of what effect 
management style has on an organization, as it is formality of process that is usually 


the key variable affected by variations in management style. 


In a small organization, where contact between decision maker and decision 
implementer is frequent, and relatively direct, a considerable amount of variation 
in formality is evident and may be tolerated; subject to tests for goal achievement. 
In large organizations, the auditor would be well advised to be skeptical of informal 
processes because of the inherent limitations of word-of-mouth communication. 
Given that sensitivity to this issue exists, tests for this potential weakness can be 


readily planned and implemented. 


It is also intuitively evident that an organization that is operated in an autocratic 
and/or centralized manner will be much more dependent on effective formal controls 
than those run on a democratic/decentralized model. This distinction should be 


reflected in audit assignment plans and programs as well. 


In conclusion, management style has an effect on an auditee organization's control 
framework which should be probed for, recognized and reflected in audit assignment 
plans and programs, if the efficiency and effectiveness of the audit process is to be 


optimized. 
Complexity of Process and Product/Service 


To those auditors well practiced in organization/process modelling the assertion 
that the control structure is directly proportional to the complexity of the system 
being modelled is self-evident. This assertion stems from the findings of sociological/ 


psychological research that all humans demonstrate a deterioration in their ability 
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to make optimum decisions as the decision increases in complexity. This topic is 
dealt with in detail in other chapters of the Handbook (e.g. Part 2, Chapter 5, "Audit 
Evidence", and Part 2, Chapter 6, "Auditor Judgment") and, therefore, will not be 


elaborated here except for the issue of optimality. 


Much of management literature reflects preoccupation with management decision- 
making, and, in particular, with the issue of maximization/optimization. Manage- 
ment science literature, typically, describes the objective of management science 


techniques as profit, maximization or results optimization. 


Ideally, the goal of maximum return or optimum result is difficult to object to; 
however, in pragmatic terms it is somewhat impractical. In the case of complex 
decisions, it is difficult, if not impossible, to enumerate all possible, relevant alter- 
natives; and even if it were, it would probably take too long. This is due both to 
inherent and human limitations on the part of the decision-maker. This situation 
naturally leads to a search for less idealistic alternatives which, although not 
abandoning the ideas of maximization/optimization where practical, recognize less 
demanding, but more viable, approaches (e.g. "Bounded rationality", March and 


Simon‘; and, "muddling through", Lindblom”). 


The auditor needs to recognize those limitations in assessing control weaknesses in 


order to ensure that results reflect realistic expectations. 


In carrying out assessments of auditee operations, auditors should expect more 
complex processes/products to be backed by more elaborate decision-support systems 
and corresponding controls. By analogy, it should not be surprising to the auditor if 


simple environments have little documentation, or none at all. 


4 March, J.G. and Simon, H.A., Organizations. 


- Lindblom, C.E., The Science of Muddling Through. 
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The Presence of Informal Coalitions 


It is well known that all organizations possess informal links and power structures 
which do not necessarily conform to the formal organization structure. This has 

already been alluded to in a subsequent sub-section dealing with the auditor's role 
as catalyst. The implications of this, of course, go well beyond the change-agent 


role. 


The effectiveness of the audit role depends heavily on its ability to influence the 
decision-making process of the auditee. This presumes knowledge of the key players 


in that process (both direct and indirect - e.g. influencers/opinion leaders). 


Actually, knowledge of the power base in the auditee/host organization is important 
to all aspects of the audit process: in planning, when deciding on key audit target 
areas; in the audit process, when accumulating audit evidence; at the reporting 
stage, when deciding on materiality of findings and the form of related recommen- 
dations; and finally during the follow-up stage, when deciding who is most likely to 


be instrumental in a recommendation being implemented. 


In sum, an effective internal audit group is well advised to identify and take due 


account of the host organization's "movers and shakers". 
8g 


Deputy Head/Superior, Audit Committee and Management Committee Relationships 


Deputy Head/Superior-Auditor Relationship 


As advocated in the Standards, (see Institute of Internal Auditors in Bibliography) 
it is preferred practice to have the head of internal audit report to the deputy head 
of the host organization. However, the relationship must go beyond simple 


organization chart formalities. 


The relationship must be "real". That is, the head of internal audit must have 
meaningful, on-going contact which is visible to the rest of the organization. 
Crucial points of contact are: during strategic and long-term planning to identify 


thrusts, concerns and priorities; prior to each major audit to identify specific 
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concerns; during the audit for urgent feedback; after the audit for debriefing of 
selected results (directly and/or through the audit committee); and ad-hoc for advice 
(e.g. regarding external audits or central agency reviews, special audit requirements) 


and administration. 


Where the deputy head is not the auditor's immediate superior, the potential for 
disruption of the preferred communication pattern, as described above, is increased. 
Where the deputy head is not the immediate superior, the preferred relationship is 
one in which the head of internal audit reports administratively to the superior 
(direct reporting relationship) and functionally to the deputy head (dotted-line 
relationship). If this situation prevails, then all relationships described above may 
be retained except for the administrative one. Unfortunately, this is unlikely. 
Where preferred reporting relationships are not implemented it is usually due to a 
large span-of-control or other factors which usurp the deputy head's time, making 


her/him unavailable to the head of internal audit as often as would be desired. 


In the case where contact with the deputy head is restricted, alternative 
organizational arrangements and channels of communication need to be established 
such that the independence, support and contact regarding organizational strategies, 


thrusts, concerns and priorities are preserved. 


Audit Committee and Management Committee Relations 


This subject is covered extensively in Chapter 2, Volume I of the Internal Audit 
Handbook, however, some elaboration will be provided here regarding the associated 
management committee activities. Associated activities, in this case, include 
discussing organizational and program strategies, thrusts and concerns, monitoring 


the organization's state and building organizational rapport. 


Where the audit committee activity is simply a unique time, or a section, in the 
agenda set aside by management committee for audit matters, the audit head can 
be given a standing invitation to remain as observer for the rest of the meeting in 
order to carry out the associated activities just discussed. However, where a 
distinct audit committee exists, then other methods may be employed to achieve 


the same ends, including an invitation to attend individual management meetings. 
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This may be accomplished by visits with individual managers (necessary, to some 
degree, in any case) and attendance at management committee meetings, at least 


as an observer. 


The advantages to being an invitee to management committee meetings, particularly 
those of senior management are: it provides an opportunity for the auditor to sample 
managerial climate, preoccupations, concerns, Coalitions, etc., first hand; and it 
provides management an opportunity to get informal audit feedback and to take 


advantage of the auditor's special skills. 


To summarize, the role of internal audit relies heavily on an intimate understanding 
of the host organization and its components. Therefore, effective reporting relation- 
ships and committee participation are indispensable to an effective internal audit 


function. 
The Client versus Auditee Issue 


In the foregoing the term "auditee" has been used indiscriminately to represent 
both client and auditee. The distinction is between the deputy head - the client, 


and the manager whose operations are actually subject to audit - the auditee. 


The distinction is useful from various points of view as far as relationships are 
concerned. Although audits are performed for the benefit of both client and auditee, 
it is important to remember which is prime and what differences in product that 


implies. 


The client is generally several organization levels removed from the operations. 
This makes judgment of true performance more difficult. This is partly due to the 
normal filtering and consolidation that goes on in hierarchical communications 
(formal and informal) and partly due to the human tendency to skew upward feed- 


back in favour of good news. 


The client, therefore, wants an independent confirmation that the auditee's area is 
performing as intended. To a large degree this translates into concerns that the 


Management Information System (MIS), whether formal or informal, indicates; i.e. 
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that the MIS is reliable and, except where executive level policy decisions are 
required, that action to correct deficiencies is being taken. On the other hand the 
auditee, being much closer to operations, has a better knowledge of the state of 
those operations. Therefore the auditee is relatively less interested in an 
independent confirmation of the state of operations and more interested in new 
insights as to the implications of the state of affairs, as reported, and identification 


and elaboration of opportunities for improvement -- a somewhat different emphasis. 


This is not to say that the client is not interested in new insights and opportunities 

for improvement. However, it must be remembered that most findings and recommen- 
dations will be acted upon by the auditees on their own, and only in the case of 
department-wide (highest level) issues will the client need to participate personally 

in the decision on action to be taken and its subsequent implementation. Both 

points of view must be understood and observed in dealings with the respective 


respondents. 


In planning for audits, although the auditee views and concerns need to be considered, 
it is the client who should have the final word - for two reasons: because of greater 
priority and because of the greater independence and, therefore, objectivity of the 


concerns expressed. 


In sum, the primary and secondary relationship of auditor to client and auditee 


respectively needs to be recognized throughout the audit process. 
The Use of Audit Agents 


The use of audit agents introduces an important added variable into an already 
complex environment. However, it should be pointed out that this added variable 


could have positive or negative effects, depending on how it is implemented. 


The positive aspect could stem from the fact that, in recognition of a complex or 
highly specialized auditee environment, the audit head contracts for highly respected 
audit agents with expertise in appropriate areas. The degree to which this turns 

out to be a positive experience will depend on the degree of prior respect, the 
behaviour of the audit agent during the engagement and the quality and delivery of 


the result. 
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This situation could just as easily turn out negatively if the conditions stated are 
not met; i.e. the audit agent is not highly reputed (as far as the auditee is concerned), 
mutual respect is not established, the agent does not demonstrate competence/ 


objectivity/empathy or the result is of low quality. 


An important concern to the head of audit is that the reputation of the departmental/ 
agency audit function rises or falls with that of the audit agent. As far as the 
auditee is concerned the audit agent represents the audit head. Since the head of 
audit will typically not be in a position to manage the audit agent team on a daily 
basis (normally both professional standards and government contracting policies 
require an arms-length relationship), it is doubly important to choose wisely in the 
first place and to start with a very carefully worded contract/mandate. Of course, 

a contract can be terminated during an audit, for cause, however this is seldom 


feasible and, in any case, by that time the damage is usually done. 


There is also the problem of relative familiarity with the auditee's operations. The 
audit agent cannot be as familiar with the auditee's activities as an in-house auditor, 
although repeated contracting with the same agent mitigates this problem somewhat. 
Given that relative unfamiliarity with the auditee's/client's organization and 
activities is inherent in the agent mode of audit, it is important that the audit head 
take all possible actions to minimize the possible negative effects of this. Possible 


actions include: 


provision of extensive advanced briefing and documentation to the agent; 
conditioning the auditee's expectations by describing what to expect; 
undertaking frequent debriefings with both agent and auditee; 


judicious follow-up; and 


decisive action if things go sour. 
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SECTION THREE: AUDIT PROCESS-RELATED RELATIONSHIPS 


Overview 


The two previous sections dealt with auditor-auditee/client relationships in general 
terms. When any of that material impinges on the discussion to follow it will not 


be repeated; however, its relevance is displayed in Table 2 for reference purposes. 


In what follows, those auditor-auditee/client relationships that are peculiar to 
various phases of the audit process will be discussed under six headings (Internal 
Audit Planning, The Audit Assignment Process, Audit Assignment and Periodic 


Reporting, The Audit Follow-up Process, Special Audits, and Liaison Activities). 


Internal Audit Planning 


Identifying Management Concerns, Assigning Risk Potential and Developing Audit 
Strategic/Long-term Plans/Annual Schedule 


As indicated in Chapter 3, Volume I of this Handbook and in a subsequent sub-section 
of Part 1, both auditee and client need to be consulted prior to, and during, the 
planning process in order to determine the nature and degree of their respective 


concerns. 


Consultation with client and auditee should result in a reporting strategy, in the 
form of reporting requirements, which in turn will influence the whole audit process, 
as far as order, frequency, timing, type and scope of audits are concerned. Although 
the auditees are consulted, the client has the final word, particularly on strategic 
and long-term planning issues. This provision becomes more important where 
differences of opinion arise with the auditee; for example, as to scope or timing of 


an audit in the auditee's area. 
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Table 2 


Relevance of Auditor-auditee Relationships 


to the Audit Process 


Audit Process Phases 


Auditor-auditee Audit Audit Periodic Follow- Special 


Relationships Planning Assignment Reporting up Audits Liaison 


Role-related 


Staff/Line X X xX X 
Historic X x X X 
Independence/ 

Objectivity X X Xx 
Audit vs. 

Investigation X X X 
Other Audit Groups X X 


Organization-related 


Stability X X 
Management Style X X X X 
Complexity X X X X 
Formal/Informal X X X 
Reporting 

Relationship X X X X X X 
Audit/Mgt. 

Committee Xx X X X X X 


Client vs. Auditee Xx X x xX Xx »< 
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Negotiating the Annual Schedule and Audit Assignment Plans 


As described earlier, both client and auditee are expected to be consulted in the 
preparation of strategic and long-term plans, with the client's views predominating. 
In the case of the annual schedules and assignment plans, negotiations would be 
expected to take place directly between the auditor and auditee, subject to 
constraints placed by the client, senior management and audit mandate/independence 
prerequisites. This means that negotiation is expected; however, its scope and 

range are limited to facilitating execution of audits without unduly disrupting the 


auditee environment, although raising of new concerns is not precluded. 


The Audit Assignment Process 


Audit Assignment Planning 


Although mentioned briefly in the preceeding discussion, assignment planning needs 
further discussion in the context of the audit assignment sub-process, as it sets the 


stage for the auditor's prime activity and largely determines its success or failure. 


It is at this stage in the audit assignment that the auditor either establishes rapport 
with, and support from the auditee, or is doomed to adversarial relations. Actions 


which can contribute to good rapport include: 


° reflecting auditee's concerns in the scope of the audit in readily discernable 


form; 


® establishing early contact with auditee to negotiate for: a convenient start 
date, convenient fieldwork periods (for multi-location audits), advance infor- 
mation, facilities for the auditors, help in assembling data for testing and 


offering briefings on the up-coming audit activities to management and staff; 


° following through with briefings and any other promises made; 


& being courteous and helpful but maintaining a professional attitude (i.e. avoid arbitrary, « 


autocratic, abrasive behaviour); 
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® arranging meetings/visits well ahead of the event; 
® not abusing privileges (e.g. use of facilities like copiers, word processing, 


EDP facilities); 


& not going on "fishing expeditions" or personal vendettas; and last but not 
least, 
® showing genuine interest in providing meaningful and substantial advice to 


the auditee, along with judicious feedback to the client. 


Review of Operational and Managerial Controls 


Although the ultimate objective is to provide an opinion on the state of management 
controls, it is inevitable that the auditor review the underlying operational controls 
as well. In the same way that the external auditor reviews financial controls, in 
order to determine the degree of reliance that may be placed on the data provided 
in the financial statements and consequently to determine the amount of testing 
required, the internal auditor reviews operational controls to determine the reliance 
that may be placed on management information provided in the Management 
Information System and on the continuity of operational processes. This provides 
the rationale to both auditor and auditee in those cases where auditors spend 


substantial amounts of time with operations staff. 


What is less obvious is the rationale for spending considerable amounts of time with 
staff officers. There are several reasons for this, some more evident than others. 
The staff officer is frequently in a float position, readily convertibie to in-line 
administrative and/or production roles in the short term. In addition, staff officers 
often take on some portion of their manager's role, although this is not recommended 


practice. 


This latter practice needs special attention since this type of situation makes it 
difficult for subordinates to determine exactly who is giving the direction (i.e. how 
seriously to take the officer's advice-cum-directive). This situation gets particularly 


disturbing when such action is taken on the officer's own initiative, as a self- 
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aggrandizement move, rather than on the request of the manager. In either case 
the auditor should be looking for evidence that all subordinates are aware of any 
delegation of decision-making to staff officers along with the limits of such 


delegation. 


Regardless of the latitude of the staff officer's activities, and their relative 
desirability, the existence of such an arrangement is of interest to the auditor, 
because the staff officer will likely be the best informed source on auditee 
operations/management, particularly with respect to unofficial/informal environ- 
mental conditions (e.g. coalitions, conventional wisdoms, local preoccupations and 
thrusts). 


Another aspect of modern internal auditing is that in most of the operations under 
audit the predetermined control model will not reflect absolute standards. The 
implication for auditor-auditee relations is very immediate and crucial. It means 
that the determination of weaknesses in the control framework will be much more 
judgmental and that, except for extreme examples, findings and conclusions will 
depend heavily on joint auditor-auditee agreement rather than being unilaterally 


and independently determined by the auditor. 


This puts a great premium on good rapport with auditees, at least key auditees, and 
on the technical and interpersonal skills, and negotiating ability of the auditor. 
Where good rapport is not established the likelihood of meaningful results, at least 
to the auditee, is doubtful. However, useful results for the client are not totally 
precluded. This depends on how formally the organization is managed. If 
documentation is sound and complete, much can be accomplished by document 


review and associated testing, regardless of the degree of cooperation obtained. 


The worst case of poor rapport involves deliberate attempts to mislead or harass 
the auditor. Apparent good rapport, however, does not preclude disruptive tactics. 
Where they occur they will likely be more difficult to detect simply because the 
auditor will be less on guard. The most difficult aspect of rapport is deducing the 
auditee's attitude. The auditee usually is aware of the auditor's mandate and senior 


management support and therefore will not likely be openly hostile. 
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Generally, regardless of the auditee's initial attitude, the auditor should encourage 
openness. Often a negative attitude can be due to a simple misunderstanding of 
the auditor's role, mandate and methods. Open discussion can bring this out and 


provide the opportunity to clear the air early in the process. 


A factor, which is pervasive in organizations is the "hidden agenda". This factor 

can be positive or negative in the context of an audit assignment process. For 
example, the auditee may try to push an idea which has been turned down by superiors 
or to further a personal end in an organizational power struggle. If a good idea is 
involved, such influence could be beneficial; but a bad idea could result in ill-will; if 

a power struggle is involved, it could be destructive. In any case the auditor should 
be forearmed and wary of this situation; forearmed with solid knowledge of the 


auditee's environment (including proposals accepted/turned down in recent months/ 


years) and wary of uncharacteristic cooperation. 


From the above it should be evident that there is a high reliance on the auditor's 
interpersonal skills, particularly interviewing, since much of the information/data 
needed to identify findings and form conclusions, or at least insights which lead to 
hard evidence, will be obtained from people. There is abundant literature on this 
subject; therefore, only a brief summary will be presented here, oriented to the 


internal audit: process. 


Because of the typical environment in which an auditor-auditee interview takes 
place (i.e. at worst hostile, at best neutral) it must be carefully prepared for and 
delicately executed. The potential image that the auditee might have of the auditor 


(see Section Two) needs to be kept in mind. 


In preparing for the interview, background documentation should be well researched 
so that the auditor is well prepared and does not waste the auditee's time; proper 
channels should be followed in setting up the interview; the objectives of the 
interview should be clear and points to be raised well thought out; and the conditions 
of the appointment should be strictly adhered to (e.g. time of arrival, time period 


allotted). 
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The interview itself should be kept on a courteous, friendly but businesslike footing. 
Questioning should be clear, concise and firm but not antagonistic. Questions should 
be a judicious mix of closed and open ended types so as to provide adequate opportunity 
for full expression where the auditee so wishes. Unclear answers should be 
summarized to the auditee to test the auditor's understanding and to prompt 
clarification if required. Provision for possible follow-up interviews (either with 
auditee or subordinates) should be discussed at this point, if likely to be needed in 
order to condition the auditee's expectations. Finally, closing remarks should 

include appropriate appreciation for time taken and a sympathetic acknowledgment 

of any inconvenience caused. A number of other useful points are raised in a paper 


by Robert G. Parker, in a recent CA Magazine article®. 


Audit Assignment and Periodic Reporting 


Audit Assignment Reporting 


In terms of auditor-auditee relations and, more importantly audit effectiveness, 
reporting is the most important of all the audit activities. The effectiveness of 
audit findings and associated recommendations has to be measured, ultimately, in 
terms of what improvement their implementation has made in the performance of 
the audited entity. This means that a lengthy sequence of events has to take place, 


some of which the auditor has little direct control over: 


perception, 
acceptance, 
intention, 

action initiation, 


implementation, and 


confirmation. 


6 Parker, Robert G., Learning Interviewing Skills for Problem Solving. 
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Most auditors have little difficulty in getting the auditee's attention (i.e. getting 
auditees to perceive a situation) since findings are usually a reflection of auditee 
data. Getting their acceptance is considerably more difficult. However, part of 
the difficulty experienced by auditors in gaining acceptance of their results can be 


traced to the early stages of the process involving perception. 


Perception of facts and opinions communicated by fellow human beings is generally 
interdependent with perception of the communicator (i.e. the perceived credibility 

of, and general receptiveness to the auditor in this case). The issues of rapport and 
credibility have been dealt with in prior sections of this chapter. A second important 
determinant of acceptance is the timing and context of the information being 
communicated. These issues have also been raised in a prior section (see Section Two) 


however some comment specific to the audit process is in order. 


Auditors have a tendency to delay reporting of results until the formal "exit 
debriefing" stage in the process. This is understandable to a degree, as the 
professional auditor wants to be as sure as possible of the facts and ensuing analysis 
before exposure of results. However, this virtue has its drawbacks. Being hit by a 
finding, particularly a major and unforseen one, without prior warning can have a 
very negative impact on acceptance. The finding may not be intuitively obvious, it 
may not fit in with conventional wisdom, it may threaten cherished beliefs/processes/ 
coalitions/even jobs, in which case the more time that the auditee has had to 
recognize, weigh, discuss, digest and, hopefully internalize the data, the higher the 
probability of acceptance. Also, early disclosure facilitates auditor-auditee synergy, 


the beneficial effect of which should not be underestimated. 


What this implies is that it is desirable to introduce findings, or potential findings, 
to the auditee as early as possible in the audit process. It does not mean exposing 
findings prematurely. It is not beneficial, for example, to stampede the auditee 
into premature and ill-considered concern or even Corrective action. A balance 
will have to be struck between the benefits of earlier exposure (seeding and 
cultivation of a desirable idea) and the possible negative effects of premature 


disclosure. 
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Other factors impinging strongly on acceptance of audit findings and/or 
recommendations include the orientation of the findings and manner in which they 
are presented, the degree to which the auditee's ideas, actions and mitigating 


circumstances are given credit and the way in which they are presented. 


Orientation of findings has to do with how constructive they are individually and 
how balanced in aggregate. "Manner" refers to the degree to which a cooperative 
process was established, the degree to which win-lose situations were avoided and 
the flexibility the auditor displayed in the wording and disposition of findings and 
associated recommendations. Although the auditor should not compromise on the 
facts underlying the finding, considerable latitude normally exists in how the finding 
is presented and who it is reported to. In the case of disposition at least five main 


possibilities exist, in ascending order of importance: 


® Minor finding - orally debriefed to the immediate auditee and not 


reported formally ("freebee"); 


& Important finding - where the immediate auditee is fully capable of 
dealing with the finding at that level of decision-making; no implications 
for higher level policy; formally reported in main body of the report but 


not brought forward to the executive summary for senior management; 


& Important finding - (as above) but with implications for other parts of 
the organization (e.g. functional groups admin./program); formally 


reported and distributed to implicated groups as well as to the auditee; 


® Major finding - policy or other implications for higher level management; 
formally reported in main body of the audit report and brought forward 


in the executive summary or a special report; and 


® Major finding of unusual urgency (e.g. fraud identified) - brought to the 


attention of appropriate levels of senior management immediately. 
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The auditee's intention and action initiation activities can, and should be influenced 
by thorough discussion of the rationale behind the recommendations, by insisting on 
an action plan and offering advice during its development and by follow-up activity. 
Follow-up activity is also the main vehicle through which implementation and 


confirmation are encouraged. 


The auditor is expected to play an additional role during implementation, particularly 
in the case of major infrastructure (e.g. program delivery process, EDP system) 
development. This activity has been termed "pre-implementation audit" and is 


dealt with elsewhere in Volume II, not including auditor-auditee relations aspects. 


From the point of view of auditor-auditee relations the potential for conflict-of- 
interest in carrying out this type of audit has to be recognized and preventive/ 
protective action taken accordingly. Two key actions come to mind. The first is 
to avoid a decision-making role in the project team (i.e. adopt the usual auditor/ 
adviser stance) and the second is to avoid the assignment of the same auditor to 


subsequent audits of the relevant auditee area for a period of time. 


Periodic Reporting 


Periodic reporting includes presentation of annual summary reports on audit 
accomplishments and of audit plans. This is dealt with adequately elsewhere in this 
Volume. However, what needs further discussion is periodic reporting which 
represents management concerns and audit findings that do not fit conveniently 


into audit assignment reports. 


The need for these has been introduced and discussed in Volume I, Chapter Three 
Development of Internal Audit Plans. In terms of auditor-auditee relations this 
vehicle is an important step in demonstrating the audit function's relevance. The 
key is structuring the content, form and timing of audit results in a way which is 
recognized by the auditee as addressing concerns in his/her terms rather than force- 


fitting them into the traditional auditor's approach to reporting. 
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The Audit Follow-up Process 


This subject is covered in other chapters in Volume II and in other sections of this 
chapter. Therefore there will be no extensive discussion of this activity here except 


to reiterate a few basic aspects of its impact on auditor-auditee relations. 


In carrying out follow-up activity, three pertinent facts need to be kept in mind: 
the manager has prime responsibility for confirmation, in particular, and follow-up 
in general; the audit group cannot be expected to follow up on all audit findings, 
regardless of importance; and repeat audits may not be sufficiently frequent to 


provide meaningful confirmation. 


Therefore, it is important to encourage the auditee to verify (confirm) that the 
change made in the auditee environment, as a result of an audit finding, has been 
designed, implemented, is being operated as intended, and that the intended results 


are being achieved. 


Special Audits 


Special audits, by their very nature, have the greatest potential for reinforcing the 
audit function's relevance and also the greatest potential for undermining its 


independence role and ultimately its effectiveness/existence. 


It is of course, always difficult to say "no" to your superior's request for service 


without endangering your organizational affiliation. 


For example, it is difficult to convince a pragmatic manager that the auditor, who 
has just spent several weeks of his/her own time plus a considerable amount of the 
auditee's time on familiarization, identification and development activities leading 
to a major audit finding, should not continue the process by implementing a solution. 
It is simply a case of the auditee capitalizing on an investment, not to mention the 


flattering argument that the auditor is the best equipped to do that. 
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Similarly, it is not difficult to see how a senior manager could come to the 
conclusion that the auditor is the best equipped to carry out an investigation of a 


suspected fraud or defalcation. 


The problem is that there are many activities/projects that are similar enough to 
auditing to be mistaken for it. In these cases the auditor has the responsibility to 
advise the potential auditee-client of the implications of the auditor taking on the 


proposed task and, if possible of viable alternatives. 


These examples are, of course, non-audit tasks and will be dealt with further in the 
next section. There are, however, many instances of legitimate special audits. 

These generally take the form of special-interest, non-standard audits whose scope, 
depth or other characteristics do not conform to one of the standard types of audit. 
Aside from the fact that they, by definition, are not of standard form and therefore 
require extra, sometimes more innovative effort, there is little about the special 
audit process that has implications for auditor-auditee relations beyond those already 


covered in previous sections. 
Liaison Activities 


To the degree that any contact with the auditee has a lasting impact on auditor- 


auditee relations all audit activities have liaison implications. 


The purpose of raising liaison at this point is to sensitize the auditor to the need 
for on-going contact (i.e. throughout the audit process) for purposes of maintaining 
a residual knowledge of managerial climate and concerns and for cultivating a 


residual rapport. 


Liaison activity can easily become very time consuming and, therefore, should not 
be overdone. However, a limited amount of such activity can pay dividends in 
facilitating relevant planning, less time-consuming audit assignment start-up and 


better auditor reception during audit activities. 
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SECTION FOUR: IMPLICATIONS OF NON-AUDIT ACTIVITIES 


In discussing the involvement of auditors in non-audit activities a distinction needs 
to be made between non-audit activities that are detrimental to future audit activity 


and those that are neutral. 


The dangers resulting from the former have already been dealt with in terms of 
potential impairment of the auditor's independence. The only danger in the latter 

is that it takes resources away from legitimate audit activity and therefore, may 

not be cost-effective. In the case of the former, not only should that use of audit 
resources be questioned but the impact on the future effectiveness of the audit 
function should be seriously reviewed as well. In either case, significant re-allocation 


of resources to non-audit activity should require the approval of the audit committee. 
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CONCLUSION 


In the foregoing an attempt has been made to sensitize the auditor to the existence, 
and implications of various behavioural effects for the audit function, and to what 


actions the auditor might take to deal with them while maintaining audit effectiveness. 


Two generic aspects of the function were considered. The first, role-related, dealt 
with those aspects that, when recognized could be dealt with by auditors through 
individual preparation or action. The second, organization-related, is infrastructure 
based and much of the action suggested, or implied, has to do with setting up 
appropriate structures or processes to start with or alternatively, recognizing the 


implications due to inherent organizational peculiarities and acting accordingly. 


This was followed by a section on auditor-auditee relations which are peculiar to 
the various phases of the audit process. Here, an attempt was made to integrate 
the generic behavioural concerns with those arising from the nature and timing of 
the activities carried out. In particular, the implications for auditor-auditee 
relations, of activities carried out early in the process (e.g. planning) and for 


auditee behaviour in later stages (e.g. reporting) were highlighted. 


Finally the potential effects of non-audit activities on auditee behaviour were 
briefly explored. The potential for loss of independence (and ultimately for 
impairment of audit efficiency and effectiveness) was dealt with at various points 
throughout this chapter while the issue of cost-effectiveness of resource utilization 


was raised in Section Four. 
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CHAPTER EIGHT 


COMMUNICATION CONCEPTS FOR INTERNAL AUDITORS 


INTRODUCTION 


Much has been written about technical competence for auditors, and the emphasis 
on audit training in government has been largely in that direction. Apart from 
technical skills, auditors must possess a variety of interpersonal skills to meet the 
demands of internal auditing successfully. Among the latter, communication counts 


high on the list of essential abilities for an auditor. 


While communication plays an integral and vital role in the performance of most 
professions, the internal auditor must be an especially skilled communicator to do 
the job properly. Communication skills form the basis for auditor-auditee relations 
and are essential to reduce the negative image which so often accompanies the 
auditor. The auditor must know not only how to communicate verbally and 
emphatically, but must be able to recognize and handle the feelings and needs of 


the auditee. 


There is ample literature on the subject of communication. This will not be 
repeated. This chapter attempts to condense many aspects of communication down 
to those which have most relevance to auditing. Some examples of communications 
requirements as applied to various steps in the audit process are given, but the 
chapter speaks more to those elements which are relevant to the auditor as a 


communicator. 
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SECTION ONE: COMMUNICATION CONCEPTS 


There are more than 25 different conceptions of communication, more than 50 
different descriptions of the human communication process and more than 15 
different models. But there are similar characteristics in all communication 


processes. 


Theories of communication include mathematical, behavioural and transactional 
models which attempt to put forward different explanations of what constitutes 
communication. For the purposes of this chapter, it is only necessary to understand 


the basic communication process which underlies all theories. 


Communication begins when a sender has a message to convey. This message is 
coded or conceptualized and transmitted verbally or non-verbally. It is decoded or 
interpreted by the receiver who then takes action based on the interpretation. The 
receiver provides feedback to the sender repeating the entire process. When one 
considers that this process occurs each time communication takes place, it is easy 
to visualize the host of expectations and perceptions which influence the way in 


which messages are sent and received. 
Communication Skills and Auditing 


Of all the interpersonal skills necessary to an internal auditor, the ability to 
communicate is probably the most essential. It is the common thread running 
through the entire audit process and affects its efficiency and effectiveness. The 
manner in which the auditor communicates with the auditee will have an impact on 
the image, professionalism, credibility and ultimate success of that auditor. Indeed 


communication skills are at the core of auditor-auditee relations. 


1 Sereno, Kenneth, and David Mortensen, Foundations of Communications Theory, . 
New York: Harper and Row Publishers, 1970. 
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Auditor-auditee relations produce endless possibilities for stress, anxiety and 
frustration. Well-developed communication skills can assist the auditor in 
overcoming or significantly reducing their effects or, preferably, preventing their 
occurrence in the first place. Ona personal level, communication skills can lessen 


the stress of interpersonal relations. 


To be an effective communicator the auditor must first know the purpose of the 
communication, whether it be simply to inform, to obtain information, to change 
ideas and opinions or to obtain agreement. When the purpose is clear, the auditor 


can proceed to plan an appropriate approach. 


Second, the auditor must know the willingness and capacity of the auditee to 
understand and accept the message. The auditor must therefore be concerned with 
the auditee's professional background, experience, level of interest and perceptions 
and recognize that these factors will affect the manner in which the message is 
received. It is unrealistic to expect all audit situations to result in total acceptance 
of the message being sent by the auditor. However, communication can still be 
effective if there is enough acceptance to get desired results. There may also be 
situations when it is not possible to persuade an auditee, and in those cases, more 


gains may be made by settling for a fair hearing. 


Third, the auditor must know the impact required to influence the auditee to action. 
Different words, methods and emphasis are required to convey positive, negative or 
neutral findings. The auditor must get and maintain auditee interest. The auditee 
must be analyzed to determine whether the auditor is dealing with hostility, 
sophistication or apathy in the audit situation. This analysis will allow the auditor 


to select the appropriate approach to urge the auditee to action. 


Last, the auditor must be able to devise appropriate methods to deliver the desired 
message with the right impact in order to influence the receiver to act. In the 
case of providing information which is fact-based, a presentation approach would 
suffice. However, changing ideas and obtaining agreement involves a whole range 
of logical and psychological considerations for the auditor. Changing ideas means 
convincing the auditee of the logic of the auditor's contention and influencing the 


auditee's willingness to listen and accept ideas. 
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In the world of the internal auditor, certain dimensions of communication are more 
relevant than others. While it is recognized that many aspects of communication 
enter into the auditor-auditee relationship, three are considered to occupy most of 


the auditor's time and therefore offer the most challenge. These are: 


8 establishing rapport; 
e eliciting information; and 
& gaining agreement (acceptance of findings and recommendations). 


It should be emphasized at this point that these three challenges are interrelated 
and on-going during the audit process and do not necessarily occur in any specific 
order. They are iterative in nature but are treated in the order presented for 


purposes of this chapter. 
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SECTION TWO: ESTABLISHING RAPPORT 


Establishing rapport represents a critical aspect of auditing since auditee cooperation 
hinges on positive auditor-auditee relations. Apart from a tense environment, poor 
audit relations can affect the progress of an audit where auditor's suggestions are 

not accepted or where information is withheld by the auditee. Moreover, studies 
have shown that when cooperation is established between auditors and auditees, 
through a participative approach, the auditor's recommendations are more readily 


implemented.” 


While an audit environment can be positive, auditors must be equipped to deal with 
a negative one. The realities of auditing all too often produce ambiguity, conflict 
and political aspects which auditors must face. Auditees may be defensive about 
having their work reviewed and being subjected to criticism, fearing change. 
Hostility may be encountered by the auditor and manifested in verbal and non- 
verbal behaviour, attempts to intimidate and, in the extreme, physical efforts to 


eject the auditor. 


Auditors may find themselves being used to further the personal goals of the auditee 


and must resist the temptation to exhibit their resentment. 


Establishing rapport in the face of such obstacles represents a real challenge to the 
auditor. Technical competence, while necessary, is not sufficient to win the respect 
and cooperation of the auditee. Interpersonal skills and an understanding of 
behavioural science and human relations are crucial counterparts in the rapport 


equation. 


The notion of rapport implies a range of factors, which, when taken together, 
produce an atmosphere of cooperation, harmony and dialogue. Achieving rapport 


requires a combination of personal qualities, experience and training. 


2 Harmeyer, James W., Golen, Steven P., Sumners, Glenn E., Conducting Internal 
Audit Interviews, May 1984, Institute of Internal Auditors Inc., p. 4. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 8, Section 2 - 590 - 


Auditors must temper a professional demeanour with a friendly attitude, exhibit 
openness and adaptability but know when to draw the line; to quote Sonja Sinclair, 
be "cordial but not cosy".? The auditor's style and approach will naturally be a 
factor in establishing rapport. Efforts must be made to avoid exhibiting either an 
authoritarian, arrogant attitude or a laissez-faire approach. Striking the proper 


balance can mean the difference in gaining the desired cooperation. 


A key component of nurturing rapport is keeping the auditee fully informed at all 
points in the audit process. Openness about the audit approach and audit findings 
will lower auditee defenses and encourage healthy dialogue between auditee and 
auditor. If possible, it is advantageous for the auditor to initiate informal contact 
with the auditee prior to the audit. Dropping by to introduce oneself can serve to 


break the ice and facilitate the more formal communication which will follow. 


The first formal point of contact with the auditee is normally during the assignment 
planning phase of the audit process. Both oral and written communication are used 

to inform the auditee of the scope, objectives and criteria as well as general audit 
strategy proposed. At regular intervals during the audit assignment, the auditors 
should inform the auditee of general findings to create an atmosphere of participation 
and to ensure there are no surprises for the auditee later when the audit findings 


are presented. 


In this approach, the auditee is able to discuss concerns and voice disagreement and 
ensure that the auditee's viewpoint is considered. Making the auditee feel a part of 


the process is key to maintaining rapport. 


There are some situations, however, when the auditor must accept the fact that 
auditee opposition will remain strong and other methods of influencing the auditee 


must be found. 


3 Sinclair, Sonja, Cordial but not Cosy; A History of the Office of the Auditor 
General, Toronto: McClelland and Stewart, 1979. 
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Eliminating or Minimizing Opposition 


Inherent in the concept of establishing rapport is the idea of eliminating or 
minimizing opposition. Indeed if the auditee is not won over in the initial stages of 


the audit process, the quality of the audit will certainly suffer. 


Eliminating or minimizing opposition becomes necessary in trying to overcome 
hostility or change beliefs. Auditees exhibiting these feelings put up barriers to 
communication which can seriously impede progress. Recognizing these barriers is 


essential if the auditor hopes to deal effectively with the auditee. 


Hostility can be aroused if auditees perceive interviews as threatening. Auditors 
must take care not to react in a defensive manner as this could compound communi- 
cation problems. Self-control and objectivity should be displayed to neutralize the 


auditee's negative feelings. 


The auditor will often find it necessary to change the auditee's opinions when the 
latter's tendency is to see either extreme of an issue to the exclusion of the middle 


ground. 


In wrestling with such communication difficulties, the auditor should accept the 
fact that there will be audit situations where it will not be possible to change 


opposition. 


There are certain techniques which are useful to the auditor in attempting to 


eliminate opposition. 


@ Seek common ground - The auditor should begin discussions with 
statements or assumptions with which the listener will agree. Later in 
the conversation, the auditor can move to the idea with which the auditee 
initially disagrees. Couching disagreement in the broader terms of 
agreement will serve to put disagreement in context and minimize the 


possibility of a negative tone. 
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Vardaman, George T., Effective Communication of Ideas, New York: 
Van Nostrand Reinhold Company, 1970. 


4 


Cite case examples - The auditor can often persuade a hostile auditee 
by marshalling examples and illustrations to support audit contentions. 
By building a convincing case to support audit opinion, the auditee will 


more likely accept the conclusion. 


Use candour - This approach can often disarm the auditee since it 
encourages an open exchange of information and assists in dispelling the 
hidden agenda impression so frequently suspected by auditees. Candour 
may take many forms. It can be achieved by attempting to reach a 
compromise or a workable solution despite disagreement, providing the 
auditee is willing to listen. Or the auditor may simply state his/her 
views along with the reasons and appeal to the auditee to rethink 


ae 4 
opinions. 
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SECTION THREE: ELICITING INFORMATION 


Up until the analysis of facts and the writing of a report, internal auditors spend 
most of their time eliciting information. The most familiar audit technique used to 
elicit information is that of interviewing. Much has been written on this subject 
and this section will not attempt to summarize the literature. Rather the aim is to 


focus on interviewing as a specialized form of communication for auditors. 


Successful communication during interviews depends on thorough preparation and 
an understanding of what auditees want and need and how they think. It requires 
training and effort. Auditors should be given the opportunity to develop and hone 
interpersonal skills through training in interviewing techniques. These human 


relations skills will determine the nature of auditor-auditee relations. 
Audit Interviews 


Audit interviews play a major role in the gathering of evidence and provide 
information to corroborate, explain or contradict information gathered through 
other sources. Sometimes interviewing is the only method of gathering evidence. 
Auditors must be especially skilled in interviewing techniques to ensure they are 


gathering the right evidence. 


Proper preparation for an interview allows auditors to become familiar with all 
available information on the person to be interviewed and the topic to be discussed. 
Preparation enhances the communication process by allowing the auditor to talk in 
an informed manner and to recognize and later deal with discrepancies in information 


which may surface during the interview. 


A second important preparatory step for interviews is the development of information 
needed by the auditor and the design of questions to elicit this information. The 
design of questions will determine, to a large degree, the nature of information 

which will be obtained. Questions should be short, clear and direct. "Yes" and 

"No" questions should be avoided. While open-ended questions may be valuable in 

that they encourage the auditee to elaborate, care must be taken so that auditees 


do not digress. If properly handled, an open-ended question can provide an 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 8, Section 3 - 594 - 


opportunity for the auditor to gain insight into significant factors impacting on the 


audit, such as the auditee's management style. 


Conducting an interview involves much more than a knowledge of its protocol - it 
requires skill in the art of listening. The auditor should ensure that a proper 
introduction is made, outlining the purpose of the interview, the topics to be 
discussed, where they fit in the audit and how the results of the interview will be 
used. A good introduction can help put the auditee at ease and minimize 


defensiveness. 


In the conduct of an interview, auditors must be cognizant of impediments to the 
communication process, be they verbal or non-verbal. Differences in perception, 
personality conflicts, poor organization of ideas, physical distractions, defensiveness 


and poor listening habits are a few barriers to communication. 
The Importance of Listening 


"Almost 50 per cent of our time involves listening." Auditors must therefore 


develop listening skills and be aware of factors which can interfere with listening. 


Active listening involves paying attention, remaining objective and rational, 
understanding the point and ensuring information is captured. Active listening 
means focusing on what is being said, restating the auditee's basic points, requesting 
clarification when points are unclear and remaining objective and unemotional. 


Note-taking is recommended to ensure that major points are not lost. 


The risk of taking copious notes is that the auditor is likely to miss what is being 
said. On the other hand, depending on memory to write notes after the fact can 
cause distortion. Either extreme has disadvantages, and the auditor should also 


take cues from the auditee when deciding on a note-taking strategy. 


> Harmeyer, James W., Golen, Steven P., Sumners, Glenn E., Conducting Internal 
Audit Interviews, May 1984, Institute of Internal Auditors Inc., p. 10. 
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One method of dealing with this matter is to have two auditors present at each 
interview where feasible. The presence of two auditors will enhance the accuracy 
of the interview exchange by allowing one auditor to conduct the interview and the 


other to record notes. 


The auditor must also be a critical listener and know what to look for and what to 
challenge. For example, a questioning style which asks for specifics rather than 
generalities from the auditee is likely to produce more pertinent information. An 
auditor must be able to identify illogical or faulty thinking often evidenced when 
conclusions are drawn from limited information, and to read between the lines 


when circumstances dictate. 


Auditee feedback, particularly non-verbal messages, provides valuable clues to the 
auditee's attitude at any given time. In fact, most communication occurs non- 
verbally and it is therefore of utmost importance for the auditor to be sensitive to 
such cues. Body language such as looking at a watch, giving a puzzled expression 
or a blank stare should alert the auditor to the need to restate or clarify the 


question, and perhaps even adapt the interview approach. 


The auditor should demonstrate sensitivity to the operating conditions of the auditee 
and know when to end an interview. If all questions have not been asked within the 
allotted time, the auditor should only continue if the auditee is willing and should 


otherwise schedule another meeting. 
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SECTION FOUR: GAINING AGREEMENT 


Gaining agreement lies at the very heart of an internal auditor's success as a change 
agent. This section examines gaining agreement through both oral and written 
means. A discussion of techniques for gaining agreement as they apply to oral 
interaction is contained in the first part of the section, followed by a review of 


written reports which constitute such an important aspect of auditing. 


Apathy and hostility are great barriers to communication in that they create either 
a complete lack of interest or auditee opposition. They may be the result of poor 
experience with a previous audit, the auditee's feeling of powerlessness or the 
perception of a threatening situation. Before a strategy to gain agreement is 
advanced, the auditor must analyze the auditee to ensure the use of appropriate 


techniques to overcome any negative feelings that exist. 


The process of gaining agreement begins subtly by establishing rapport. The better 
the rapport, the higher the probability of gaining agreement. However, rapport 
may be excellent yet the auditee may express disagreement. There are two notable 
stages when gaining agreement becomes critical: at the front end - gaining 
acceptance of audit criteria, and at the exit interview - gaining acceptance of 


audit findings. 


Gaining Acceptance of Audit Criteria 


The ultimate success of the audit hinges on initial agreement to a management 
control model acceptable to both auditor and auditee. For many audit areas, criteria 
to judge the adequacy of controls have not been translated into generally accepted 
principles. This may become the crux of a problem in that the auditee may not 
accept the criteria as the basis of an assessment of operations. The auditor may 
encounter greater difficulty where the nature of the operation is oriented towards 
innovation and creativity; managers of such areas may be inclined to view their 


operations as unique and management control as a hindrance. 
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In a scenario where the auditor has developed appropriate audit criteria but the 
auditee is not in agreement, the auditor must be prepared to use both logical and 
psychological arguments to convince or "sell" the auditee on their adequacy. 
Logical arguments might include reference to generally accepted practices as an 
authoritative source and to early discussion and agreement with auditees on their 


application to auditee operations. 


Chapter 2, Part 2, Volume II of this Handbook discusses, in detail, the development 
of predetermined control models and audit criteria. Auditors are referred to this 
chapter to gain an appreciation of how the logic inherent in this process might be 


presented to an auditee to gain agreement. 
Gaining Acceptance of Audit Findings: Oral Debriefing 


The job of persuading a critical auditee in a debriefing requires an open, rational 

and logical presentation of facts and opinions. An orderly and complete presentation 
will allow the auditee to scrutinize the logic and assess the validity of the auditor's 
statements. It should be remembered that the purpose of the exit interview is to 
provide an opportunity for the auditor to test conclusions and for the auditee to 


comment, raise questions and voice concerns prior to issuing the written report. 


The auditor must make clear the objectives of the communication - what is to be 
demonstrated. This will focus the discussion for the auditee. The auditor must 


first state what has been assumed or what has been the starting point. 


The audit scope should be stated as well as any limitations or exclusions which 
would place conditions on that scope. The procedures and methodology used by the 
auditor to analyze and reason through the data collected should be stated, followed 


by a description of the findings, conclusions and their significance. 


Although there should be no doubt about the facts accumulated, and working papers 
should support them, auditors should be open about possible weaknesses in their 
ideas and opinions (e.g. such as being tentative or even faulty, as well as the reasons 
for this), and should acknowledge difficulties or constraints experienced during the 


audit. It is not unusual for an auditor to couch certain findings in terms of 
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limitations such as unavailability of resources or time to pursue an issue in greater 
depth, particularly issues of intermediate materiality. For example, in a case where 
a judgmental sample has been used, the auditor would only be able to comment on 


the sample and not the whole population. 


The auditor must be receptive to critique by the auditee and be prepared to deal 
with challenges to statements made. Testing of the auditor's ideas in this manner 
will ensure that the auditee is able to voice concerns. In fact, the auditor should 
solicit criticism in the interests of the best thinking, and to avoid the possible 


impression that matters are being concealed. 


Based on auditee feedback, the auditor may need to rethink ideas or opinions and 
perhaps change others. If the interchange between auditor and auditee is cooperative 


and participative, the prospects of gaining agreement will be brighter. 


The other aspect of gaining agreement, that is the psychological or human aspect, 
is more concerned with the impact of the auditor's message on the auditee. Aside 
from delivering a logical message, the auditor must be capable of controlling the 


reaction to the message so that the auditee is willing to listen and accept ideas. 


Two approaches for gaining agreement are 1) moving from the familiar to the 
unfamiliar, and 2) moving from the acceptable to the initially unacceptable. In 
these approaches, the auditor begins with information or ideas the auditee feels 
comfortable with and builds an understanding of new ideas, suggestions or 
recommendations the auditee may be hesitant about or opposed to. The auditor 
should of course, use facts to convince the auditee, but examples, comparisons, and 
analogies can assist in the development of ideas and lend more credibility and 
persuasive power. Where disagreements are irreconcilable, the auditor should state 
that the facts must be reported as seen by the auditor, but that the views or 


opinions of the auditee will be included in the report. 


In some situations, the exit interview may be supplemented with a visual presentation 
utilizing flip charts or slides. Such a technique is suitable when a group of interested 


departmental officials is to be debriefed or when special emphasis is desired. 
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The auditor's goal, in the end, is to move the auditee from belief to action. 
Without this outcome, gaining agreement becomes academic. If the auditee is 
ready, action may be induced through a subtle or direct challenge. More often a 
plan of action is solicited whereby the auditee can spell out what will be done to 
correct problems. 


Report Writing 


Report writing merits special treatment since it is this medium on which the 
auditors depend to communicate formally the results of their work and represents 
the evidence of their analysis. The report is the tool used to inform, persuade and 


advise others on ways to improve and provides a record for follow-up. 


Auditors write audit reports to win agreement and effect change. Therefore not 
only is it important to express ideas clearly and accurately, but the auditor must 
find the right approach to report writing - that is, one which is suited to the 


particular reader or readers. 


Clarity and Precision 


The clear and precise expression of ideas begins with an understanding of the 
purpose of an audit report which must be clearly defined at the outset. The auditor 
must weigh the need to which the report should be "selling" proposals against the 
extent to which such proposals have already been sold. The nature of recommen- 
dations must be considered and whether special emphasis (graphs, charts, etc.) 


should be added to clarify or explain them. 


To assist the auditor in deciding on the arrangement of the report, consideration of 
who will use the report and how it will be used is essential. In the federal govern- 
ment, report recipients can cover several echelons. For example, the auditee will 
always receive the report, and it is normally at this level that corrective action is 
initiated. However, the auditee’s superiors will also be interested in the results of 
an audit as one means of keeping apprised of the auditee's performance. Managers 
responsible for policy that the auditee might be expected to implement will also 


be interested in audit findings, particularly where these point to policy as a problem 
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area. These managers may be superiors of the auditee, program policy advisers or 


functional managers, (e.g. finance, personnel). 


Once the auditor has clarified these issues, the structure and style of the report 


can be determined. 


Report Structure 


Report structure can vary from audit to audit, and there is an abundance of literature 
on the topic. Rather than prescribe any particular style, this section will focus on 


general principles which should be observed in writing audit reports. 


Audit reports should describe the area audited, the scope of audit activities, the 
work which was performed, an analysis of the state of the area audited, a list of 
findings or areas for improvement, (including cause and effect/impact analysis), 
audit conclusions along with their rationale and audit recommendations. The addition 
of an executive summary will often prove helpful to readers who are farther removed 
from auditee operations but nonetheless have an interest in the report. Appendix | 


illustrates a recommended outline for an audit report. 


Secause the success of an audit report depends on its credibility, it is critical that 
findings are substantiated by factual evidence. Even one unfounded statement ina 
report can destroy its credibility and jeopardize good rapport with the auditee. 
Therefore the auditor should ensure that source data are double checked and that 
there is careful analysis and scrutiny of audit findings so that conclusions cannot be 


diminished. 


Scrutiny of an audit report should occur as part of the quality control function in 

an internal audit organization. Quality control would typically involve a critical 
review of the report contents including language, tone, logic and length. This review 
will cover an examination of language for completeness, conciseness, appropriateness 
and accuracy. Words must be chosen carefully to get the reader to read, understand 
and act on an audit report. Tone must be carefully considered so that the report 
does not appear overly critical or fault-finding, but rather leaves the reader with a 


more accurate impression of the state of the auditee operation. 
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Much of the persuasive impact of a report is accomplished through logic. The auditor 
must reflect a clear-thinking approach to the justification and promotion of 
proposals made. But despite the adequacy of language and logic, the length of a 
report may deter the auditee from a careful review. There is no ideal length for an 
audit report, but it should be short enough to maintain the reader's interest, yet 
complete enough to cover all relevant information. Mechanical faults such as poor 
spelling, incorrect syntax and messy appearance must be completely eliminated 


from the report so as not to diminish its value. 


Depending on the nature of the audit, the auditor may be faced with the complexities 
of multi-level reporting and how best to present findings under such circumstances. 
In an organizational audit of an administration unit for example, the auditor will be 
reporting on all responsibility centres within that unit. These would typically 

include personnel, finance, communications, etc. There is no standard or universal 


model for audit reporting in these situations, but there are preferred options. 


One option is to report by letter or memo to all responsibility centre managers 
outlining specific findings pertaining to their individual areas of activity. The 
letter may or may not be accompanied by the formal report, as deemed appropriate. 
Advantages of the letter or memo include its ability to add a personal touch, to 
acknowledge special courtesies which may have been extended and to focus findings 
to the specific recipient. Brevity is recommended and the letter should not 
normally exceed two pages. These letters can be adjoined to the formal report for 


the primary recipient's information. 


Alternatively, the report may be written in sections corresponding to the various 
responsibility centres, which can be extracted as required, for transmittal to the 


manager concerned. 


The concept of separate letters or memos is applicable for matters of lesser 
importance that should be put in writing to the auditee but are not of sufficient 


importance to be included in the report to the deputy head. 
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An additional report, different in nature from the audit report, yet a vital instrument 
in informing the deputy head of the progress of the internal audit function is the 
annual report to the deputy head. The head of internal audit should submit a report, 
outlining the actual audit coverage as compared to the annual schedule, along with 
explanations of significant deviations. Major audit findings and recommendations 
during the year and recurring issues should be summarized as well as the status of 
corrective actions. The annual report is also an opportunity to raise other matters 


requiring the attention of the deputy head. 


Report Style 


Determining an appropriate style for an audit report requires analyzing the environ- 
ment and personal characteristics of the recipient. The auditor must try to talk in 
the reader's language and address the report to the reader's interests. Auditees are 
more knowledgeable and experienced in their field than the auditor is likely to be 
and the auditor should confine statements to what can be demonstrated through 
standards, tests and procedures rather than venture opinions which relate to the 


auditee's professional competence. 


Auditors are often at a disadvantage in terms of gaining agreement through a report 
because the auditee is generally defensive and skeptical before the reading even 
begins. To counteract this, auditors can influence more effectively by adopting a 
style of complete reporting. This style promotes balance by acknowledging the 
achievements of the auditee's organization (e.g. good practices or action already 
initiated), as well as the shortcomings and gives credit where it is due. Such an 
approach tends to lower the auditee's resistance to the audit report and increase 


receptivity to findings and recommendations. 


Written communication must be complemented by oral communication in the form 
of continuous dialogue between auditor and auditee to facilitate acceptance of the 
report. This contact will alert the auditee to probable recommendations before 


they appear in print and will reinforce the constructive orientation of the audit. 
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It is not the role of the auditor to order or demand action, although this can be 
quickly forgotten by auditors focusing on compliance issues. The emphasis or tone 
must be on assisting the auditee in attaining operational objectives. Even though 
the auditees usually have little choice in the matter of whether they will be audited, 


an audit report must be "marketed" to the auditee to pave the way for change. 
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SECTION FIVE: COMMUNICATIONS AND THE INTERNAL AUDIT ORGANIZATION 


Aside from communications with auditees, there are significant communications 
impacts within internal audit organizations stemming from heads of internal audit 
and audit managers which must be recognized and managed. This section examines 
specific communications aspects at different levels in the internal audit organization 


and their potential ramifications. 


Role of Heads of Internal Audit in Communications 


The head of internal audit is a key player in the establishment of an effective 
internal audit organization. While organizational structures certainly set the 
foundation for the role of internal audit in a department or agency, the communi- 
cations ability of the head of internal audit can greatly advance the image of the 
organization. There are at least four areas of work in which the head of internal 
audit can be influential in promoting the respect, professionalism and credibility of 
the internal audit group. These are in managing the organization, in advising the 
departmental audit committee, in liaising with management, and in liaising with 


external audit organizations. 


Managing the Organization 


Successful communication in the management of the audit organization can be 
difficult to achieve given the general orientation of the audit management team 
towards upward communication. Auditors may have little contact with the head of 
internal audit and, as a result, they may not understand the demands placed on that 
job or on audit managers. Auditors may have unclear ideas about what superiors 

are actually doing and therefore develop the sense that superiors are more concerned 


with "politics" and unimportant tasks. 


If no action is taken to dispel such misconceptions, informal grapevines can emerge 
creating rumours and exaggerations. Communications of this nature can be positive 
in terms of free exchange of ideas and transfer of information, but they can also be 


destructive to the organization by breeding discontent and morale problems. 
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Keeping in touch with audit staff formally and informally can alleviate these 
problems. Practices such as regular staff meetings where audit staff can be exposed 
to the issues affecting the organization and can express concerns assist communi- 
cation. To some degree, performance appraisal interviews can assist in providing 


Opportunities for improved communication between supervisors and employees. 


Advising the Audit Committee 


The audit committee provides the internal audit group with a communication vehicle 


to the deputy head and can thus increase the status and authority of auditors. 


The head of internal audit frequently acts as secretary to this committee and can 
influence the proceedings by having input to the agenda, organizing material for 
committee members, formulating lines of enquiry and providing technical assistance. 
The better the communication skills of the head of internal audit, the more influence 
he/she is likely to exert. The degree to which the audit committee is able to assist 
deputy heads in fulfilling their responsibilities for the performance of their 
organization is largely a function of the quality of audit issues raised and the 
communication skills of the head of internal audit in clarifying and elaborating on 


these issues. 


Liaising with Auditee Managers 


Liaising with auditee managers is an on-going communication activity aimed at 
keeping abreast of the evolution of auditee organizations, operations and delivery 


systems. 


Liaising with External Audit Organizations 


The head of internal audit will, from time to time, become involved in liaison with 
external auditors such as the Office of the Auditor General, the Public Service 
Commission or the Commissioner of Official Languages and quasi-audit organizations 
such as program evaluation, departmental review groups and central agency review 
groups. The purposes of this contact would typically include resolution of audit 
issues, coordination to avoid duplication and work-sharing arrangements with the 
external auditor. 
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Through skillful communication, the head of internal audit can achieve cost- 
effective audit effort, smooth working relationships and can make progress towards 
the resolution of the audit reliance issue. These issues can be delicate since the 
practice, for example, of placing reliance on the work of internal audit, is not well 
established and there are no universal guidelines governing the conditions under 


which internal audit work shall be judged to be reliable. 
Role of Audit Managers in Communications 


Audit managers normally find themselves in the middle of the communication flow 
in an internal audit organization. They are the recipients of communication from 


heads of internal audit on the one hand and from auditors on the other. 


Depending on their organizational relationship to the head of internal audit, audit 
managers may be in frequent contact or may rarely see the head of internal audit. 
Even if contact is frequent, audit managers may still experience difficulty translating 


information downward. 


Communication skills are a significant factor in the quality of leadership and 
tutorship audit managers provide to their auditors. Key ingredients in the success 
formula of an audit group are motivation, team spirit and commitment. Building 
these attributes is central to improving productivity. Audit managers can achieve 
these objectives by being straightforward and open with auditors. Employing 
communications techniques which make auditors feel included in decision-making 


and support auditors' efforts will inspire loyalty and confidence. 


Audit managers depend on auditors for the quality of information reflected in an 
audit report. There is a vested interest in ensuring that communications are 
adequate to ensure that what is reported is accurate and fair. This becomes all the 
more important for audit managers who do not spend much time in the field. 
Coaching to supplement professional practices and to provide constructive criticism 


can help motivate auditors. 


Being accessible to auditors and taking an interest in their career development 


contributes to an atmosphere of support and encouragement. 
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CONCLUSION 


In the foregoing, an attempt has been made to illustrate the types of 
communications challenges facing the internal auditor and how the auditor should 


respond to maximize effectiveness. 


Communications requirements were discussed in terms of establishing rapport, 
eliciting information and gaining agreement. Various techniques for achieving 


communications objectives were examined, covering both oral and written methods. 


Section Five elaborated on the written medium of reports, covering particularly 
important aspects of report writing such as structure and style and emphasizing the 


need to gear presentation to the interests of the auditee. 


The last section dealt with the communications aspects at work within the internal 
audit organization and the respective roles of the head of internal audit and audit 


managers in ensuring effective communications links. 
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Appendix | 
FORMAT FOR AUDIT REPORTS 
1. Covering Letter 


- identifies the audit, audit time period and initial distribution of the 


report. 
2s Table of Contents 
ai Executive Summary 


- where required, summarizes the major observations, conclusions and 


recommendations for the ease of the deputy head. 
4, Scope and Objectives 


- contains a description of the audit entity to give the reader insight into 


the internal and external factors influencing auditee operations; 

- outlines audit objectives and scope as well as any limitations on scope 
to avoid the impression that audit work was conducted where it was 
not; 

- outlines methods and procedures followed in the conduct of the audit; 

~ acknowledges the assistance provided by the auditee staff. 


aa Summary of Observations and Conclusions 


- provides a summary of major observations and conclusions as they relate 


to the audit objectives; 
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: gives clear explanation of conclusions including the basis on which they 


were formed; 


- outlines any corrective action which has been taken or which may be 


required on the part of the auditee; 
- should not create an overly negative total impression; 
- should not exceed two pages. 
Detailed Audit Observations and Recommendations 


- sets out specific observations and make recommendations for corrective 


action; 


- includes comments on the extent and effectiveness of corrective action 


taken in preceding audits; 


- should not include the names of individuals responsible for adverse 


conditions; 


~ sets out the problem and provides all facts related to allow the reader 


to understand the issue, including materiality; 
- sets out the causes of the problem, assessment of the risk of the problem 
and action required, as well as the level of management responsible for 


corrective action; 


- may include details of the audit evidence where such adds to the reader's 


comprehension of the issue; 
- may identify good practices which have application to other areas; 


- should include only observations of major concern, (observations of 


lesser importance may be dealt with orally or through a separate letter). 
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fe List of Recommendations 


- recapitulates recommendations made with references to appropriate 


pages in the report; 


- should provide space for the auditee's response and target date for 


completion. 
8. Appendices 


- may contain charts, tables, graphs, etc., to support the text of the 


report; 


- provides explanatory but expendable information. 


Szi-98 
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CHAPTER NINE 
AUDITOR RELIANCE 
INTRODUCTION 


The purpose of this chapter is to sensitize the internal auditor to the issues relating 
to the concept of reliance, the various conditions under which reliance can occur 
and the general criteria that might apply and the special conditions that auditors 


should be aware of in the various reliance situations. 


At this point in time, criteria for reliance are quite general, thus necessitating 
considerable judgment by the auditor in the determination of the degree of reliance 


and subsequent reduction in the extent of audit undertaken. 


Considerable research is necessary in order to improve the criteria for reliance for 
the various reliance situations and to validate the reliance process. It is hoped that 
current research being carried out by the Canadian Comprehensive Audit Foundation 
and the deliberations of the CICA's Public Sector Accounting and Auditing 


Committee will contribute guidance that will help to improve current practice. 


The presentation in this chapter begins with a discussion of general criteria for 
reliance (Section One), followed by a discussion of reliance on internal control 
(Section Two) and ending with a discussion of the conditions of relevance when 
considering various types of reliance, including reliance on specialists (Section 
Three), reliance on other reviewers/evaluators (Section Four) and reliance on other 


auditors (Section Five). 
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SECTION ONE: GENERAL CRITERIA FOR RELIANCE 
What is Reliance? 


According to Webster's New World Dictionary, to rely is to trust, depend on, or 
have confidence in someone or something. Applying the dictionary meaning into 
the audit context, the auditor will rely on work done by another party if it meets 


certain criteria. 


The purpose of reliance is generally improved efficiency (i.e. lower cost) in 
completing the audit, however, it may have other benefits as well. For example, it 
may result in more timely reporting, more complete coverage or new insights. The 
reliance process may also be used as a means for on-the-job training; for example, 
in the case of joint audits or the secondment of a member of one audit group to 


another. 


In audit circles the term reliance has been generally applied to the relationship 
between the external auditor, as the one relying and other external auditors, 
internal auditors or specialists, as those being relied upon. However, the concepts 
involved are equally applicable to reliance by internal auditors on the work of 


others. 


The key issue in reliance is that of clear, relevant, meaningful and credible criteria 
that are sufficiently precise to be objectively measurable and, therefore, not 
susceptible to judgmental biases or errors. In the past, biases have occurred in 
both directions; relying where reliance should not have occurred, and the more 
common situation, that of external auditors not relying when it was feasible and 


cost effective to do so. 
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In the following, criteria will be discussed in two stages. First, those criteria which 
are independent of who is being relied on will be dealt with. This will be followed 


by a discussion of those criteria that are unique to the party to be relied on. 
General Criteria for Reliance 


In order to treat criteria for reliance in an orderly way and to provide context for 
the detailed discussion to follow, it may be useful to begin by relating criteria for 
reliance to the audit process. ! Specifically, criteria for reliance will be categorized 
according to whether they pertain to the examination and assessment of the infra- 
structure (compliance procedures) or results (substantive procedures) of the party 


to be relied upon (see Table 1). 


The implication here is that the process of deciding on the extent of reliance is like 
an audit. That is, in deciding on criteria for reliance, the auditor "wears the 
auditor's hat" and approaches the task in the same way that he/she would approach 


that same task in any other audit situation. 


Keeping in mind the nature of procedure or test the auditor intends to perform, i.e. 
compliance or substantive, Table 2 provides an elaboration of both internal and 
external audit general criteria that may be used to determine the degree to which 


an auditor may rely on an internal and external auditor respectively. 


—— 


1 For a recent discussion of this approach see Ronald G. Peters, "The 
External/Internal Auditor Relationship - Part 1," CA Magazine, March 1984. 
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Table | 


Reliance Determination 


High-level Internal Audit External Audit 

Criteria” Process? Process’ 2 
Objectivity, Audit of Compliance 
Competence Infrastructure Procedures 


(Delivery Systems) 


Results Audit of Substantive 


Results Procedures 


2 CICA Handbook, Auditing Recommendations, 5215.16, Canadian Institute of 
Chartered Accountants, Toronto. 


3 Internal Audit Handbook, Volume II, Part 2, Chapter 1, "Internal Audit 
Approaches". 


4 *CICA Handbook, Auditing Recommendations, 5215.15. 
5 *Internal Audit Handbook, Volume II, Part 2, Chapter 5, "Audit Evidence". 


*(The Internal Audit Handbook employs similar terminology but in different 
context, given the differences between financial attest and internal auditing.) 
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High-level Criteria 
(External Audit)® 


Objectivity 


Competence 


Results 
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Table 2 


General Reliance Criteria 


Detailed Criteria 
(External Audit)® 


Reporting level 
(reporting administratively) 
Reporting level 


(reporting results) 


Experience 
Qualifications 
Hiring, training, 


supervision practices 


Nature of recommendations 


Detailed Criteria 
(Internal Audit) 


Organizational reporting 
relationship 

Audit policy 

Audit committee 

Training 


Supervision 


Audit policy and plans 
Knowledge and experience 
Training and supervision 
Audit process, 

methods and techniques 


Use of specialists 


Nature of findings 
Adequacy of evidence 
Adequacy of cause- 
effect analysis 
Adequacy of recommendations 
Adequacy of auditee's 
response and action plan 
Adequacy of plan 


implementation 


6 CICA Handbook, Auditing Recommendations, Section 5215. 
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The criteria for reliance on internal audit are taken largely from the Performance 
Assurance Review (PAR) Guide’. A more complete list is provided in Appendix A. 
The source of the external audit criteria is the CICA Handbook, Section 5215. More 
detailed criteria for reliance of external on internal auditors are suggested in 


8 and Spronck? (please refer to the Bibliography). 


articles by Peters 
It will be readily seen from the criteria lists in Table 2 that there is still 
considerable leeway for the use of judgment in deciding on the degree to which the 
actual state or performance of the party being assessed for potential reliance 


satisfies the respective critieria. 


Implicit in the assessment process is the assumption that the detailed criteria used 
will be very much dependent on the nature and purpose of the desired reliance, and 
that the explicit criteria are augmented by the professional qualifications and 
requirements of the assessor. For example, when an external auditor is assessing 
the competence of another external auditor, the degree of conformance to criteria 
would tend to correlate strongly with the degree to which the characteristics (e.g. 


educational background) of the party being assessed resembles that of the assessor. 


In the case of financial auditing, given the structure of public accounting in Canada, 
the norm would generally be the CA, however, in the case of comprehensive auditing 


and special examinations the situation is less clear. 


In the next four sections these aspects of reliance will be explored. 


7 A guide developed for internal use by the Office of the Comptroller General, 
Policy Development Branch, Internal Audit and Special Studies Division (based 
on the Standards for Internal Audit in the Government of Canada, Treasury 
Board of Canada (Comptroller General) 1982). 


8 Ibid (see footnote 1). 


9 Managing Coordinated External and Internal Audits, Chapter 5, 
Lambert H. Spronck, John Wiley & Sons, 1983. 
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SECTION TWO: RELIANCE ON INTERNAL CONTROL AND INTERNAL AUDIT 
Reliance by the External Auditor 


The literature on reliance of external auditors on internal control (including internal 


audit) is extensive, !% 11 


and will not be repeated here. However, there are aspects 
of this form of reliance which bear further discussion. These are (1) the degree of 
reliance that actually takes place, (2) the usual criteria actually used to decide on 
degree of reliance on internal audit, and (3) the issue of reliance in the case of 


external audits which are not attests of financial statements. 


Degree of Actual Reliance 


In the past there has been general agreement among external auditors that 
examination of internal controls, in order to help determine the extent of testing, 
is a good thing. The idea of reliance has an obvious attraction from the point of 
view of audit efficiency. There is, however, less general agreement as to how 
much the results of internal control review should influence the auditor's decision 


on extent of testing. In terms of criteria, this is the weakest link in the chain. 


In a recent survey (experiment)! a aimed at determining agreement among auditors 
on (1) judgments concerning the reliability of an internal control system and 

(2) resulting decisions on the extent of substantive testing (sample size), it was 
found that although there was a high correlation on the first point, there was not 
the same degree of agreement on the second, i.e. the selection of sample sizes. 
Generally it was found that auditors tended to choose about the same size of sample 


regardless of how reliable they judged the internal control system to be. 


10 CICA Handbook, Canadian Institute of Chartered Accountants. 

11 Tabor, Richard H., "Internal Control Evaluations and Audit Program Revisions: 
Some Additional Evidence," Journal of Accounting Research, Volume 21, No. I, 
Spring 1983, U.S.A. 

12 Ibid (see 11). 
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Aside from being a reflection, to some degree, of the inherently conservative nature 
of auditors, this survey result indicates that more research is needed in this area. 
This issue is, of course, considerably complicated in the case of comprehensive/value- 
for-money/broad-scoped auditing, where the internal controls of interest include a 
much wider range of controls than in the case of a financial attest audit, where 


interest is focused primarily on financial controls. 


The Usual Criteria Used to Decide on Degree of Reliance on Internal Audit 


Consistent with the general case of internal control, there is much more agreement 
regarding reliance "on paper" than there is in practice. On paper, there is most 
agreement at the relatively abstract level of "competence, objectivity and results" 
as being good criteria for judging reliance. The difficulty is that these criteria are 
too vague and subjective. A recent article in the Internal Auditor !?, in referring 
to the Statement on Internal Auditing Standards No. 9 (SAS No. 9), issued by the 
American Institute of Certified Public Accountants (AICPA), said "SAS No. 9 sets 
general standards for reliance on internal auditors. I use the term general because 
the standards are broad-based, open-ended, and vague at times. ...the generality of 
these terms has made effective reliance assessments difficult." The above comments 
are equally applicable to the criteria provided in the CICA Handbook, although the 


Handbook does provide a more detailed level of breakout (see Table 2). 


The survey described in the above-mentioned article and its companion from the 
Journal of Accounting Research! * indicated that the two criteria that auditors 


agree on most are "independence" and "previous year's audit work". 


13. Brown, Paul R., "New Evidence on the Reliance Judgment," The Internal 
Auditor, October 1984. 


14 Brown, Paul R., "Independent Auditor Judgment in the Evaluation of Internal 
Audit Functions," Journal of Accounting Research, Vol. 21, No. 2, Autumn 
1983. 
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Here again, experimental results point to the need for more research. As in the 
case of reliance on internal control, there is insufficient reliable data on the 
correlation between reliance criteria being met and actual reliability of results 
(produced by internal controls or the internal audit function respectively). If one 
could establish a high degree of correlation between specific characteristics of 
internal control (or internal audit) and reliability of their respective output (results) 
there would be much more inclination by auditors to carry that knowledge forward 


into "extent of testing" decisions. 


Reliance in the Case of Broad-scoped Auditing 


The issue of criteria for reliance becomes considerably more complicated in the 
case of broad-scoped auditing simply because the scope of the auditor's activity 
goes, typically, beyond financial matters and is carried on in non-financial areas of 


the audited organization. 


In this situation, even the rule-of-thumb of using the skill set of a typical external 
audit team as a model for evaluating "competence" becomes problematic since 
there is little agreement on what skills, beyond the traditional audit skills, are 
needed to carry out such an audit assignment. Opinions vary between "this is an 
audit like any other audit", at one extreme, to "an accountant is not competent to 


carry out a non-financial audit" (e.g. personne! audit) at the other. 


The answer seems to lie somewhere in between. Clearly, to carry out an audit 
takes audit skills. The issue is whether broad-scoped auditing requires only an 
augmentation of subject matter expertise (a reliance issue in its own right -which 
will be dealt with in Section Three) or an extension of the range of basic auditing 
skills, which equip the auditor or audit team with the ability to examine and assess 
non-financial as well as financial operations. As discussed elsewhere in this 


Handbook, it appears that both are required. 


In the short term there is no generally accepted solution to this problem aside from 
the use of judgment, supplemented by the guidance provided herein. In the longer 
term, some help may be on the way with at least two professional bodies taking 


initiatives which could throw light on the subject. These bodies are the Canadian 
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Institute of Chartered Accountants', Public Sector Accounting and Auditing 
Committee, which intends to deal with the general subject of comprehensive 
auditing; and the Canadian Comprehensive Auditing Foundation, which is also 


carrying out relevant research in this area. 
Reliance by the Internal Auditor 


Given the purpose, scope and specific objectives of internal auditing, reliance on 
internal control is as simple, or as difficult, as the internal audit process itself. 
This is because, unlike external audit, examination and assessment of internal 
control is the main thrust of internal auditing, rather than simply a means of 


reducing the extent of substantive testing to be performed. 


This does not make the job any easier for internal auditors than for external 
auditors, but internal auditors do not have the option of not relying. At the 
minimum a control not relied upon would have to be documented, reported upon 


and substantiated, in terms of the impact on organizational operations of its lack. 


Since, as already noted above, reliance on internal controls is the heart of internal 
auditing, the process of direct reliance is covered by other chapters in Volume II, 
Part 2 of this Handbook and will not be dealt with here. Reliance on indirect 
internal controls, such as monitoring, inspection, quality assurance, program 


evaluation and similar functions will be discussed in Section Four. 


Internal Audit Handbook 
Volume II, Part 2 
Chapter 9, Section 3 - 623 - 


SECTION THREE: RELIANCE ON SPECIALISTS 


There are basically two ways in which specialists may be assigned to an audit team. 
One is as an auditor with at least dual expertise, that is, the usual audit background 
plus an added skill, say EDP; the other way is as a specialist, for example a 
Statistician. 


In the first case there is no reliance problem since the auditor with additional 
specialization easily meets the minimum requirements of the ideal model. Any 
minor variations in performance would be dealt with as a normal part of the audit 
supervision process. The second case is what is normally referred to as "reliance 


on a specialist". 


In Section 5360 of the CICA Handbook !? there are several examples of situations 
where specialists might be used. This list is equally applicable to internal auditing, 
however, it would need to be considerably augmented to reflect the broad-scoped 
auditing situation. In attest auditing, the use of specialists is an exception, while 
in broad-scoped auditing, an audit not requiring the use of specialist skills would be 


the exception. 


The CICA Handbook also provides factors that will influence the auditor's decision 


as to the extent of his/her procedures as follows: 


® the materiality of, and the risk of significant error in, the item being 


examined in relation to the financial statements as a whole; 
@ the complexity of the item; and 


& the absence or nature of other source of audit evidence available with 


respect to the item. 


15 Canadian Institute of Chartered Accountants. 
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In addition, the three general criteria mentioned earlier - competence, objectivity 
and results - are also relevant, although in a narrower sense, commensurate with 


the scope of the specialist's involvement. 


In the case of "results" or output, although relevant criteria may exist, their direct 
verification by the auditor may be problematic, as the auditor may not have the 
technical background to judge conformance (e.g. judging the quality of output of a 
statistician, econometrician, industrial engineer, geologist). The more usual approach 
to judging achievement of results is to do it indirectly through inquiry as to reputation. 
This effectively combines the results criteria with that of "competence". Auditing 
recommendation .09, in Section 5360 of the CICA Handbook, employs the following 


wording to cover these two criteria: 


.09 When the auditor plans to use the work of a specialist as audit evidence, 
he/she should have or obtain reasonable assurance concerning the 


specialist's reputation for competence. 


The "objectivity" criteria is somewhat more complicated to apply. It depends on 
both the nature and scope of input expected and on the allegiance of the specialist. 
The key distinction that must be made as to scope is whether or not the specialist 
will participate in the evaluation phase of the audit. If yes, then the issue of 


objectivity becomes a considerably more important criteria. 


As implied above, there are two broad categories of employment possible for 


specialists. The first is procedural and therefore less sensitive. This would include: 


2 advice to the auditor in the assignment planning stage, particularly in 


the development of lines of inquiry; 


° advice to the auditor in developing the pre-determined control model 


and associated audit programs; and 


@ advice to the auditor in setting up verification tests. 
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In the above cases, although objectivity is as important as in all auditing assignments, 
independence is less of a concern. It must be remembered that the specialist will 
be working under direction of a senior auditor, as an adviser not as a decision- 


maker. 


Where the specialist is utilized in the evaluation phase, independence becomes 
crucial as the supervising auditor is, presumably, not in a position to evaluate fully 
the results on his/her own. In this case, reputation takes on more importance as 
well. In this respect, dealing with specialists who are "professionals", that is they 
are subject to professional standards and ethics, is an advantage. The degree of 
the advantage depends on the credibility or reputation of the profession concerned, 


the organization for which the specialist works, and that of the individual specialist. 


Regarding allegiance of the specialist, the following list ranks some typical 


possibilities in order of increasing independence: 


® auditee staff on temporary assignment to the audit group (i.e. staff 


seconded from the group being audited); 


® specialists* on loan from other groups in the auditee's organization for 


one assignment (i.e. not from the auditee group being audited); 


® specialists* seconded to the audit group for developmental purposes; 
® specialists* permanently assigned to the audit group; 
e non-professionals on contract; 
cs professionals on contract. 
* As has already been indicated, specialists who are also "professionals" tend to 


have more credibility, to the degree that their profession has credibility. 
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SECTION FOUR: RELIANCE ON OTHER REVIEWERS/EVALUATORS 


The criteria for reliance, discussed in Sections One, Two and Three, all apply here 
to some degree, depending on the circumstances. To the degree that management 
relies on reviewers or evaluators as part of its control framework, they fall into 

the category of internal control; to the degree that these functions are performed 
by specialists (e.g. industrial engineers, for quality control; classification experts, 
for classification audit; economists/social scientists, for program evaluation), the 


criteria discussed in Section Three apply. 
Activities that fall into the "other reviewers/evaluators" category are as follows: 


- Monitoring or Review Groups 

- Inspection 

- Quality Assurance/Control 

- Auditors in specialty areas (e.g. classification, staffing, materiel) 


- Program Evaluation 


Aside from criteria for reliance, of equal concern to the internal auditor are the 
logistics of reliance. To rely on output from such groups, certain procedural measures 


need to be taken: 


® provision needs to be made (e.g. in audit programs) for utilization of the 


output from such groups as evidence; 


& internal audit resourcing procedures should provide for possible utilization 


of specialists from such groups in the audit assignment; 


5 internal audit methodology should incorporate their methods and 


techniques where relevant and beneficial; and 


® internal audit should synchronize its long-term plans with those of review 


or evaluation groups in order to maximize the potential! for relying. 
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Two of the above mentioned review/evaluation groups are discussed in more detail 
below; review groups that act as extensions of the internal audit group with varying 
degrees of functional direction and control and quasi-audit groups, whose history 


generally precedes that of broad-scoped internal auditing. 
Decentralized Review Groups 


This is the case of head office auditors relying on decentralized reviewers, (sometimes 
called auditors). Although similar standards and criteria apply as for the case of 
internal auditors relying on specialists, there is a further consideration: functional 


direction. 


If an internal audit group is in the position of having to rely regularly on a review 
group within the same organization (e.g. head office internal audit group relying on 
a regional review group that reports to the regional manager), then it would be 
more efficient to "institutionalize" the relationship. That is, the relying group 


would be well advised to negotiate prior agreement on some or all of the following 


criteria: 

e ‘agree to work to a common mandate (or agree that the subsidiary review 
group will cover off some pre-determined sub-set of the mandate, 
including such issues as purpose, scope, frequency and reporting regime); 

e agree to work to common standards and ethics; 

@ agree on the skill (knowledge and experience) requirements of the 
reviewers; 

& agree on methods and techniques; 

é agree on integrated, or at least coordinated long-term plans; 

e agree on the scope, objectives, lines of inquiry and reporting requirements 


for review assignments; and, most importantly 
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° agree on the right of the head office audit group to review the subsidiary 
group for adherence to the terms and conditions of the reliance agreement, 
complete with provision for sanctions in the case of non-compliance. 

The first part of this requirement is inherent in the mandate of the 


internal audit group, however, the second may not be. 


(This review could, of course, be performed by the head office group 
itself or contracted for; there could also be reliance on the external 


auditor.) 


The above institutionalized criteria or conditions for reliance amount essentially to 


what is commonly referred to as a "functional" relationship. 
Reliance of Internal Auditors on Other Auditors 


This is in reference to those cases where there are groups of specialized auditors in 
an organization (e.g. in personnel: classification, staffing and official languages 
auditors; EDP auditors that operate outside the internal audit group; contract 
auditors; security auditors). Although these groups are often termed auditors, they 


are generally treated as review/monitoring groups by internal auditors. 


These cases are problematic from the reliance point of view as, in most cases, they 
do not meet internal audit independence and competence criteria. On the other 
hand, these auditors are generally more knowledgeable in the auditees' area and 
carry out much more detailed audits/reviews than could ever be justified by an 


internal audit group. 


Reliance decisions in these cases would depend on the usual competence/objectivity/ 
results criteria, recognizing the inherent limitations on independence for such 
groups, as they usually report to the auditee. However, even if full reliance (e.g. 

on results) is not feasible, it may be beneficial to rely on their documentation and 


advice at the assignment planning and review stages at the least. 
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SECTION FIVE: RELIANCE ON OTHER AUDITORS 


Reliance of auditors on other auditors is, on the face of it, the easiest case. However, 
there are several possible complications. For example, auditors have varying degrees 
of auditing education and/or experience. This is particularly true in the case of 
broad-scoped auditing. Secondly, internal and external auditors have differing 
mandates; this limits the degree of reliance to something less than 100 per cent. 


Thirdly, organizational proximity will have a bearing on reliance decisions. 


In the following, various situations where internal auditors might rely on other 


auditors are examined. 


Reliance on External Auditors 


In this case the issues of competence and objectivity are usually of minor significance 
since external auditors tend to be the standard or normative model, at least in the 
area of.financial auditing. However, even in the area of financial auditing there is 
a considerable difference between performing an attest audit on financial statements 


and an internal financial audit; the common ground being internal financial control. 


When making reliance decisions in the case of broad-scoped audits in non-financial 
areas, the reliance issue gets considerably more clouded. Here the criteria discussed 
in previous sections, particularly those of competence and results, need to be applied 
in the same way that they would be applied by the external auditor when deciding 


on reliance on internal auditors. 


As in all cases where standards and related criteria are an issue, mutually agreed 
terms and conditions would be a great asset. In this regard, the research being 
conducted by the Canadian Comprehensive Audit Foundation, on the subject of 


reliance, should be beneficial. 
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In any case, heads of internal audit should become familiar with the intentions of 
external auditors regarding their organizations and make every effort to reflect 
such activities in their audit plans. In the ideal case, the results of external audits, 
particularly comprehensive audits, may be relied upon, thus, eliminating the need 
to audit areas of the audit universe covered by them; and vice-versa, it should be 
possible to convince the external auditors that they can rely on internal audit work 
where scope and areas of coverage overlap, thus, eliminating double coverage in 


those areas. 
Reliance of Internal Auditors on Other Internal Auditors 


Reliance of internal auditors on other internal! auditors is both easier and more 


difficult than reliance on external auditors. 


On the one hand, all internal auditors operate under the same standards (at least 
theoretically), have similar mandates and scope (usually), and use similar methods 
and techniques (more or less - this is a question of degree). On the other hand, the 
internal audit population is much less homogeneous (uniform) as to educational 


background, ability and experience. 


In any case, the standards provide a relatively broad basis for reliance decisions 
(see Appendix A at the end of this chapter). This Handbook should also be helpful, 


to the degree that it elaborates on the standards. 


The most obvious areas of reliance by internal auditors on other internal auditors 
are the areas where one organization provides services to another organization. In 
the federal government a prime example is Supply and Services Canada, which 


provides a number of services including cheque issue and accounting and purchasing. 


In these cases, it is assumed that the internal auditor in the user department or 


agency would rely on the internal audit of the service supplying department. 
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Another area of possible reliance is where several departments are obtaining similar 
services from a common private sector supplier (e.g. computing services vendor). 

In this case it would be advantageous for the heads of internal audit concerned to 

get together and either delegate one of the internal audit groups to act on behalf 

of all, form a joint group for the purpose, or contract with a private sector firm to 
perform the audit on their joint behalf. A fourth possibility is reliance on an external 
audit of the service centre, where the scope and results of such an audit cover the 


internal auditor's concerns. 


This, of course, presupposes that provision was made for such an audit in the 


respective contracts between the vendor of the services and the users. 
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CONCLUSION 


In this chapter, general criteria for reliance were explored, followed by discussions 
of special considerations associated with various combinations of those doing the 


relying or being relied upon. 


The process of deciding on the degree of reliance is subject to the use of considerable 
judgment, as criteria developed thus far are lacking in precision and are of undeter- 
mined validity. This is evidenced by the fact that although auditors have agreed in 
the past on the utility of reliance as a concept, they have not agreed on the degree 
to which reliance should affect the subsequent extent of testing to be performed by 


the relying auditor. 


Given the above caveats, the main benefits of this chapter are likely to be sensitization 
to the issues, greater awareness of the similarities, differences and special consider- 
ations associated with the various reliance situations that an auditor is likely to 
encounter and a better acquaintance with the genesis and current sources of reliance 


criteria. 
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Appendix B 
GUIDE FOR PERFORMANCE ASSURANCE REVIEWS: CRITERIA FOR EVALUATION 


At this point in time, our approach to evaluation of an audit function is much like 
that taken by the Institute of Internal Auditors (IIA). Basically the assumption 
made by the IIA is that, by-and-large, if the internal audit function conforms to the 
"Standards for the Professional Practice of Internal Auditing" and the auditors 
performing the audits abide by the "Code of Ethics", the function can be judged as 
satisfactory. The Office of the Auditor General is taking essentially the same 


approach as well. 
We are currently experimenting with additional tools and techniques in order to 
give the review more breadth and depth, however, these tools and techniques have 


yet to be proven. 


The following general criteria are being applied in using the methods, tools and 


techniques as outlined in the previous section: 
(a) In assessing conformance to standards, three levels of evaluation are used: 
is Existence of the required structure, process or behaviour; 


Ze Enumeration of the required elements of the required structure, process 


or behaviour; and 


Be Overall judgment of the adequacy of the required structure, process or 


behaviour. 


This is supplemented by specific evaluation guidelines published by Central 


Agencies as follows: 
- "Staffing Audit and Review Guide", PSC, April/80; 


- "Audit questions on Administrative Policies", TBC, 19380; 
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"Audit Guide on Official Languages", TBC, 1979. 


(b) A similar approach is taken to assessing the "Management of the Function", 


supplemented by: 


1. General management guidelines as reflected in various OCG and 


Treasury Board Canada Circulars, Guidelines, etc.; 


2. Literature of Management regarding what constitutes good 


management practice; 


3. Work sheet for tracking the Audit process (see Appendix G - criteria 


self-explanatory); and 


4, Work sheet for longitudinal analysis of audit results (see Appendix H). 


(c) The Institute of Internal Auditors! Standards and Code of Ethics are used as 
reference, as is the Framework for Evaluting an Internal Audit Function, 
developed by the Foundation for Auditability Research and Education and 
based on the IIA Standards. 
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